Document libFuzzer on Windows.

[metzman]

Summary

Document that libFuzzer supports Windows, how to get it,
and its limitations.

[metzman]

• mention support may be added

[metzman]

I think libFuzzer on Windows is solid enough now to make public.
What do you think?

cc @rnk

[metzman]

llvm/docs/LibFuzzer.rst

652–662

I don't understand LLVM's release process to know when the official LLVM build goes up and what parts of libFuzzer on Win it will support.
So the second sentence may not make sense.
Maybe we should wait for it to come out, see if enough parts are supported and then recommend people use the official build of 8 instead of a 9 snapshot.

[metzman]

llvm/docs/LibFuzzer.rst

652–662

when the official LLVM build goes up

I mean the official release of LLVM 8.

[kcc]

do we have the bot?

[metzman]

In D57597#1381039, @kcc wrote:

do we have the bot?

It gets built on this bot (proof is this test failure).
I think @rnk discussed enabling it on the sanitizer-windows bot. But I forgot to do this.
I can do it now if you'd like.

Or did you mean a dedicated bot, like sanitizer-x86_64-linux-fuzzer which I can also do.

[kcc]

any reliable bot will do.
This one, ... hm, doesn't seem reliable.
Last build passed two days ago :(
A separate bot would be nice if you expect a high rate of changes (and breakages) on windows

[rnk]

In D57597#1381064, @kcc wrote:

any reliable bot will do.
This one, ... hm, doesn't seem reliable.
Last build passed two days ago :(
A separate bot would be nice if you expect a high rate of changes (and breakages) on windows

I am taking steps to add it to sanitizer-windows here, which is super fast and reliable (for a sanitizer bot =P):
http://lab.llvm.org:8011/builders/sanitizer-windows

I think this doc change is ready to go as is. It reflects the state of the very near future.

llvm/docs/LibFuzzer.rst

652–662

We'd have to merge this to the 8.0 branch, but committing as normal is the first step. We'd also want to put a release note in llvm/docs/ReleaseNotes.rst on the 8.0 branch. @hans is the person to ask about this.

[rnk]

In D57597#1381590, @rnk wrote:

In D57597#1381064, @kcc wrote:

any reliable bot will do.
This one, ... hm, doesn't seem reliable.
Last build passed two days ago :(
A separate bot would be nice if you expect a high rate of changes (and breakages) on windows

I am taking steps to add it to sanitizer-windows here, which is super fast and reliable (for a sanitizer bot =P):
http://lab.llvm.org:8011/builders/sanitizer-windows

To follow-up, check-fuzzer passed on sanitizer-windows:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/41419

[99/102] Running Fuzzer unit tests
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:32: note: lsan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:39: note: msan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:57: note: linux feature unavailable
-- Testing: 36 tests, 16 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Testing Time: 0.92s
  Expected Passes    : 36

Do we need more targets?

[metzman]

In D57597#1381683, @rnk wrote:

In D57597#1381590, @rnk wrote:

In D57597#1381064, @kcc wrote:

any reliable bot will do.
This one, ... hm, doesn't seem reliable.
Last build passed two days ago :(
A separate bot would be nice if you expect a high rate of changes (and breakages) on windows

I am taking steps to add it to sanitizer-windows here, which is super fast and reliable (for a sanitizer bot =P):
http://lab.llvm.org:8011/builders/sanitizer-windows

To follow-up, check-fuzzer passed on sanitizer-windows:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/41419

[99/102] Running Fuzzer unit tests
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:32: note: lsan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:39: note: msan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:57: note: linux feature unavailable
-- Testing: 36 tests, 16 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Testing Time: 0.92s
  Expected Passes    : 36

Do we need more targets?

check-fuzzer is the only target but it seems like only the unittests are running.

llvm/docs/LibFuzzer.rst

652–662

Thanks. Will do.

[metzman]

In D57597#1381689, @metzman wrote:

In D57597#1381683, @rnk wrote:

In D57597#1381590, @rnk wrote:

In D57597#1381064, @kcc wrote:

any reliable bot will do.
This one, ... hm, doesn't seem reliable.
Last build passed two days ago :(
A separate bot would be nice if you expect a high rate of changes (and breakages) on windows

I am taking steps to add it to sanitizer-windows here, which is super fast and reliable (for a sanitizer bot =P):
http://lab.llvm.org:8011/builders/sanitizer-windows

To follow-up, check-fuzzer passed on sanitizer-windows:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/41419

[99/102] Running Fuzzer unit tests
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:32: note: lsan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:39: note: msan feature unavailable
llvm-lit.py: C:/b/slave/sanitizer-windows/build/llvm.src/projects/compiler-rt/test/fuzzer/lit.cfg:57: note: linux feature unavailable
-- Testing: 36 tests, 16 threads --
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Testing Time: 0.92s
  Expected Passes    : 36

Do we need more targets?

check-fuzzer is the only target but it seems like only the unittests are running.

Sorry, I was wrong.
The bot is running the integration tests too. I don't think we need any more targets.
Thanks @rnk !

@kcc
Now that we have a more reliable bot, does this look good to land?

[hans]

llvm/docs/LibFuzzer.rst

652–662

Sounds good to me. I'll merge this when it lands (maybe I'll drop the part about Clang 9), then please write something for the release notes. You can either send a patch for docs/ReleaseNotes.html on the branch, or just write some text and I'll add it.

[metzman]

llvm/docs/LibFuzzer.rst

652–662

Great. Will do!

[metzman]

• Make Kostya's suggested change

[metzman]

• mention /INCREMENTAL can't be used.

[metzman]

Kostya asked me offline to change the sentence about clang 8 to say what revision support was added in before landing.

[kcc]

LGTM

[metzman]

• make it clearer that we wont support incremental linking

[Dor1s]

You may need to use double backticks to put something into monospaced font, see https://reviews.llvm.org/rL352890. Maybe that's why the docs are still not updated.

[hans]

In D57597#1394264, @Dor1s wrote:

You may need to use double backticks to put something into monospaced font, see https://reviews.llvm.org/rL352890. Maybe that's why the docs are still not updated.

Added that in r353809.

Merged this to the release_80 branch in r353811.

[metzman]

In D57597#1394264, @Dor1s wrote:

You may need to use double backticks to put something into monospaced font, see https://reviews.llvm.org/rL352890. Maybe that's why the docs are still not updated.

Thanks for pointing this out. Judging from the current docs, I think single backticks for non-links just does italics.
I'm not sure what's going on with the documentation page and why hit hasn't updated though.

[metzman]

In D57597#1394411, @hans wrote:

In D57597#1394264, @Dor1s wrote:

You may need to use double backticks to put something into monospaced font, see https://reviews.llvm.org/rL352890. Maybe that's why the docs are still not updated.

Added that in r353809.

Merged this to the release_80 branch in r353811.

Thanks Hans!
I can write the release notes patch soon.

llvm/docs/LibFuzzer.rst

 
 Second, there is a subtle technical reason not to rely on the rest of LLVM, or
 any other large body of code (maybe not even STL). When coverage instrumentation
 is enabled, it will also instrument the LLVM support code which will blow up the
 coverage set of the process (since the fuzzer is in-process). In other words, by
 using more external dependencies we will slow down the fuzzer while the main
 reason for it to exist is extreme speed.
 
-Q. What about Windows then? The fuzzer contains code that does not build on Windows.
+Q. Does libFuzzer Support Windows?
 ------------------------------------------------------------------------------------
 
-Volunteers are welcome.
+Yes, libFuzzer now supports Windows. Initial support was added in r341082.
+Any build of Clang 9 supports it. You can download a build of Clang for Windows
+that has libFuzzer from
+`LLVM Snapshot Builds <https://llvm.org/builds/>`_.
+
+Using libFuzzer on Windows without ASAN is unsupported. Building fuzzers with the
+`/MD` (dynamic runtime library) compile option is unsupported. Support for these
+may be added in the future. Linking fuzzers with the `/INCREMENTAL` link option
+(or the `/DEBUG` option which implies it) is also unsupported.
+
+Send any questions or comments to the mailing list: libfuzzer(#)googlegroups.com
 
 Q. When libFuzzer is not a good solution for a problem?
 ---------------------------------------------------------
 
 * If the test inputs are validated by the target library and the validator
   asserts/crashes on invalid inputs, in-process fuzzing is not applicable.
 * Bugs in the target library may accumulate without being detected. E.g. a memory
   corruption that goes undetected at first and then leads to a crash while