[mlir-reduce] Add doc for usage of mlir-reduce

[jpienaar]

Quick pass 🙂

mlir/docs/Tools/MLIRREDUCE.md

1 ↗

(On Diff #349815)

Could the filename of the doc match the file name of the tool?

5 ↗

(On Diff #349815)

Let's consistently capitalize MLIR

5 ↗

(On Diff #349815)

trigger bugs? Unexpected behavior feels passive :-) We want to be able to reduce the size of a reproducer for a bug.

12 ↗

(On Diff #349815)

One of the largest ones (the largest?) is simply deleting code not required to reproduce an error. Which is done very heuristically/without maintaining any semantics. I think that's important to have early that it's not an optimization, this changes the program in a way that is "wrong" but useful.

14 ↗

(On Diff #349815)

Let's explain interesting right after.

[Chia-hungDuan]

Addressed review comments

[jpienaar]

mlir/docs/Tools/mlir-reduce.md

6

An MLIR input ? (it need not be a Module)

8–9

describes mlir-reduce, a tool that can reduce the size of the input needed to trigger the error.

11

This whole document is about MLIR reduce, so I think you can remove this heading

13–14

s/.../including/ ?

16

s/a module/an input/g (just to be more general, any top level input could work)

16–17

s/.../is interesting, e.g., exhibits the characteristics that you would like to focus on/ (shorten and combine with next sentence)

19

s/.../invocation fails/ ? (just a little bit more general but I think specific enough still)

21–22

I feel this could go close to the start, just before the last sentence in first paragraph.

24

Lets tag this section with a heading "Available reducer passes" or some such

I would consider doing an example invocation here to show usage before explaining all the different reducers

26

You may have to clarify here what you mean with redundant. I believe it is meant as extraneous to interestingness (e.g., we can remove them and the case remains interesting, rather than they are redundant to the semantics of the input)

27

s/way//

28

s/.../as long as it/

28

Is this true now? I thought we now ran verification too, so the output is still valid.

34

It isn't necessarily that one knows this, it is more that the pattern will be attempted and if it doesn't retain the problem, it will just be discarded (e.g., it would naturally not be explored further wherever the pattern does not retain the interestingness)

40

s/abundant// :) (or potentially we could just say MLIR does not have a fixed set of dialects)

40

In a way all is custom and so nothing is custom. There aren't 2 tiers here, how its defined is uniform. So I'd drop this. Or you could say "dialect specific rewrite patterns"

63

Perhaps, you can freely intermix regular optimization passes (that retain semantics) with reducer specific passes?

79

s/hint/specify/ ?

95

[for follow up] That is a bit surprising. Could we change that? standard in or positional argument is how mlir-opt and the other tools work. Being able to pipe the result from a tool into reduce would be nice too (but I don't know if the input file is too hard coded).

99

And this is only needed if 1) one uses custom syntax, 2) failure is due to verification in dialect, 3) there is dialect-specific reducer patterns?

118

reduce

121

You mean somewhere on disk? Perhaps lets rephrase this, the feature we want is that it should produce the most optimal result thus far when interrupted.

124

OOC why does this matter? As long as it is interesting model, the tool just trying all the things seems OK (potentially inefficient if we know some won't help, but not sure beyond that)

[Chia-hungDuan]

mlir/docs/Tools/mlir-reduce.md

26

Agree, the redundant may be confusing. Removed

28

Yes, mlir-reduce now only accepts/outputs valid file. Explained in down below, "To avoid that...". Just want to remind that the user also needs to provide valid input.

40

Removed. Yes, I think it'll be simpler.

63

Yes, I'm thinking of merging the two passes into one, this may be easier for the user's perspective(only choose the tree traversal strategies, don't bother with the set of regular optimization passes)

I'm planning that now and will have solution soon. We can keep the doc in sync with it during the review.

95

Revisiting this part, will update this afterwards

99

Add the explanation of when we need a custom one.

124

I was uncertain for this one, was thinking the case like, we knew that some patterns won't work then we can exclude them for getting shorter reduce time.

Now I think we should let the user as easy as possible to use it. the item is removed.

[Chia-hungDuan]

Addressed review comments

[Chia-hungDuan]

Remove description of -opt-reduction-pass which will be removed soon.

[jpienaar]

Nice, thanks!

mlir/docs/Tools/mlir-reduce.md

7

"we" or perhaps even better "developers"

[Chia-hungDuan]

Addressed review comment and fixed the syntax

mlir/docs/Tools/mlir-reduce.md

+# MLIR Reduce
+
+[TOC]
+
+An MLIR input may trigger bugs after series of transformations. To root cause
+the problem or help verification after fixes, developers want to be able to
+reduce the size of a reproducer for a bug. This document describes
+`mlir-reduce`, which is similar to
+[bugpoint](https://llvm.org/docs/CommandGuide/bugpoint.html), a tool that can
+reduce the size of the input needed to trigger the error.
+
+`mlir-reduce` supports reducing the input in several ways, including simply
+deleting code not required to reproduce an error, applying the reducer
+patterns heuristically or run with optimization passes to reduce the input. To
+use it, the first thing you need to do is, provide a command which tells if an
+input is interesting, e.g., exhibits the characteristics that you would like to
+focus on. For example, you may want to see if `mlir-opt` invocation fails after
+it runs on the certain MLIR input. Afterwards, select your reduction strategy
+then `mlir-reduce` will do the remining works for you.
+
+## How to Use it
+
+`mlir-reduce` adopts reduction-tree algorithm to reduce the input. it generates
+several reduced outputs and do the further reduction in between them according
+to the tree traversal strategy. The different strategies may lead to different
+result and different time complexity. You can run as
+`-reduction-tree='traversal-mode=0'` to select the mode for example.
+
+### Write the script for testing interesting
+
+As mentioned, you need to provide a command to specify `mlir-reduce` which case
+you're interesting. For each intermediate output generated during reduction,
+`mlir-reduce` will run the command over the it, the script should returns 1 for
+interesting case, 0 otherwise. The sample script,
+
+```shell
+mlir-opt -convert-vector-to-spirv $1 | grep "failed to materialize"
+if [[ $? -eq 1 ]]; then
+  exit 1
+else
+  exit 0
+fi
+```
+
+The sample usage will be like, note that the `test` argument is part of the mode
+argument.
+
+```shell
+mlir-reduce $INPUT -reduction-tree='traversal-mode=0 test=$TEST_SCRIPT'
+```
+
+## Available reduction strategies
+
+### Operation elimination
+
+`mlir-reduce` will try to remove the operations directly. This is the most
+aggressive reduction as it may result in an invalid output as long as it ends up
+retaining the error message that the test script is interesting. To avoid that,
+`mlir-reduce` always checks the validity and it expects the user will provide a
+valid input as well.
+
+### Rewrite patterns into simpler forms
+
+In some cases, rewrite an operation into a simpler or smaller form can still
+retain the interestingness. For example, `mlir-reduce` will try to rewrite a
+`tensor<?xindex>` with unknown rank into a constant rank one like
+`tensor<1xi32>`. Not only produce a simpler operation, it may introduce further
+reduction chances because of precise type information.
+
+MLIR supports dialects and `mlir-reduce` supports rewrite patterns for every
+dialect as well. Which means you can have the dialect specific rewrite patterns.
+To do that, you need to implement the `DialectReductionPatternInterface`. For
+example,
+
+```c++
+#include "mlir/Reducer/ReductionPatternInterface.h"
+
+struct MyReductionPatternInterface : public DialectReductionPatternInterface {
+  virtual void
+  populateReductionPatterns(RewritePatternSet &patterns) const final {
+    populateMyReductionPatterns(patterns);
+  }
+}
+```
+
+`mlir-reduce` will call `populateReductionPatterns` to collect the reduction
+rewrite patterns provided by each dialect. Here's a hint, if you use
+[DRR](../DeclarativeRewrites.md) to write the reduction patterns, you can
+leverage the method `populateWithGenerated` generated by `mlir-tblgen`.
+
+### Reduce with built-in optimization passes
+
+MLIR provides amount of transformation passes and some of them are useful for
+reducing the input size, e.g., Symbol-DCE. `mlir-reduce` will schedule them
+along with above two strategies.
+
+## Build a custom mlir-reduce
+
+In the cases of, 1. have defined a custom syntax, 2. the failure is specific to
+certain dialects or 3. there's a dialect specific reducer patterns, you need to
+build your own `mlir-reduce`. Link it with `MLIRReduceLib` and implement it
+like,
+
+```c++
+#include "mlir/Tools/mlir-reduce/MlirReduceMain.h"
+using namespace mlir;
+
+int main(int argc, char **argv) {
+  DialectRegistry registry;
+  registerMyDialects(registry);
+  // Register the DialectReductionPatternInterface if any.
+  MLIRContext context(registry);
+  return failed(mlirReduceMain(argc, argv, context));
+}
+
+```
+
+## Future works
+
+`mlir-reduce` is missing several features,
+
+*   `-reduction-tree` now only supports `Single-Path` traversal mode, extends it
+with different traveral strategies may reduce the input better.
+*   Produce the optimial result when interruped. The reduction process may take
+a quite long time, it'll be better to get an optimal result so far while an
+interrup is triggered.