This is an archive of the discontinued LLVM Phabricator instance.

[NPM] Print '-passes' compatible string for built pipeline.
ClosedPublic

Authored by markus on Aug 18 2021, 6:16 AM.

Details

Summary

In response to https://lists.llvm.org/pipermail/llvm-dev/2021-July/151987.html and what was discussed in that thread.

Proposal for pipeline printing with the new pass manager with integrated printing of pass parameters for those passes that implement it (currently only SimplifyCFG).

Examples

$ opt -enable-new-pm=1 -adce -simplifycfg -o /dev/null /dev/null -print-pipeline-passes
-passes='verify,function(adce),function(simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts>),verify,NO-PASSNAME-FOR(BitcodeWriterPass)'
$ opt -O3 /dev/null -S -o /dev/null -print-pipeline-passes
-passes='verify,annotation2metadata,forceattrs,inferattrs,function(lower-expect,simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts>,sroa,early-cse,coro-early,callsite-splitting),openmp-opt,ipsccp,called-value-propagation,globalopt,function(mem2reg),deadargelim,function(instcombine,simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts>),inliner-wrapper,globalopt,globaldce,elim-avail-extern,rpo-function-attrs,require<globals-aa>,function(float2int,lower-constant-intrinsics,loop(loop-rotate),loop-distribute,inject-tli-mappings,loop-vectorize,loop-load-elim,instcombine,simplifycfg<bonus-inst-threshold=1;forward-switch-cond;switch-to-lookup;no-keep-loops;hoist-common-insts;sink-common-insts>,slp-vectorizer,vector-combine,instcombine,loop-unroll,transform-warning,instcombine,require<opt-remark-emit>,loop-mssa(licm),alignment-from-assumptions,loop-sink,instsimplify,div-rem-pairs,simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts>,coro-cleanup),cg-profile,globaldce,constmerge,rel-lookup-table-converter,function(annotation-remarks),verify,print'

Diff Detail

Event Timeline

markus created this revision.Aug 18 2021, 6:16 AM
markus requested review of this revision.Aug 18 2021, 6:16 AM
Herald added a project: Restricted Project. · View Herald Transcript
bjope added a subscriber: uabelho.Aug 18 2021, 8:50 AM
bjope added a comment.Aug 18 2021, 9:04 AM

One thing that we probably need to deal with is that we still have some non-unique mappings from class name to pass name. Solution is to make the passes take params, or to use different class names when the passes should appear as different passes. If for example the DEBUG_TYPE is common it could be better to use the parameterized solution, but maybe it is a bit weird with the sanitizers that can be run as both function and module passes if that should be a parameter (it would for example be illegal to do "function(msan<module>", so those passes are maybe a bit problematic unless we use different class names for the module and function pass).

Here is a list of classes that are used with two or more different pass names (making getPassNameForClassName unreliable):

HWAddressSanitizerPass: hwasan<kernel;recover>, hwasan
ModuleInlinerWrapperPass: inliner-wrapper-no-mandatory-first, scc-oz-module-inliner, inliner-wrapper
LoopExtractorPass: loop-extract-single, loop-extract
ModuleAddressSanitizerPass: kasan-module, asan-module
EarlyCSEPass: early-cse-memssa, early-cse
EntryExitInstrumenterPass: post-inline-ee-instrument, ee-instrument
LowerMatrixIntrinsicsPass: lower-matrix-intrinsics-minimal, lower-matrix-intrinsics
AddressSanitizerPass: asan<kernel>, asan
MemorySanitizerPass: msan, msan-module
ThreadSanitizerPass: tsan, tsan-module
MemorySanitizerPass: msan, msan-module

Another thing that we should deal with is to make some framework for passes with params, making it possible to print the used params. Right now the printed string isn't trustworthy as we don't even print "passname<...>" to indicate that the pass takes params that missing in the printout.

The point of this is to reduce pass pipelines right? And this is only relevant when there's state not captured in the IR.
Another approach would be to bisect where some state becomes corrupted. We can split the pipeline into two, where we first run the first set of passes, dump the IR, then only run the second set of passes on the dumped IR. Something similar to opt-bisect. This should be much less invasive than this proposed patch.
And I have no proof of this, but it seems like we wouldn't be able to reduce a lot of the pipeline, since earlier passes, especially when running on unoptimized IR, will make a lot of necessary changes to trigger issues.

If we do go down this proposed patch's route, we need to decide to what level do we want to make a pipeline completely representable as a string. Looking at PassBuilder.cpp, I don't think it's feasible to represent literally every possible pipeline as a string.
The legacy -debug-pass=Arguments decides to give up on handling pass params. It'll be a tradeoff of representing more vs more code changes/invasiveness. If we want to support pass params of any kind, each pass will have to map its params to a string (i.e. it can't be a static method). I'm not sure I really like adding more to PassInfoMixin.
Currently we just have a mapping of names to a pass. This patch is basically trying to add a map going back the other way and also trying to keep it in sync with the PassRegistry.def map. It seems wrong to me... I never liked PassInstrumentationCallbacks::getPassNameForClassName(), I only added it to support --print-before/after.

If we're not going to use pass instrumentation, we shouldn't use PassInstrumentationCallbacks to get the pass name. We should do something similar where we gather up class -> pass names by including PassRegistry.def.

Honestly I'm tempted to manually create a string representation that closely mirrors -O3 and try to keep it up to date. We can attempt to reduce that pipeline.

markus edited the summary of this revision. (Show Details)Aug 19 2021, 1:43 AM

The point of this is to reduce pass pipelines right? And this is only relevant when there's state not captured in the IR.

Yes, that and being able to see what passes are included in a pipeline without having to run it on any special input. Keeping in mind that different targets have their own extension points so pipelines do differ depending on the target.

Another approach would be to bisect where some state becomes corrupted. We can split the pipeline into two, where we first run the first set of passes, dump the IR, then only run the second set of passes on the dumped IR. Something similar to opt-bisect. This should be much less invasive than this proposed patch.

Perhaps, I don't know how that would work though so I cannot comment on how invasive it would be nor if it would work sufficiently well. In general though if automation fails it is always useful to be able to step in and do things manually so automatic bisection would not replace that.

And I have no proof of this, but it seems like we wouldn't be able to reduce a lot of the pipeline, since earlier passes, especially when running on unoptimized IR, will make a lot of necessary changes to trigger issues.

Not sure about that. If the IR is already reduced on a full pipeline (with e.g. llvm-reduce) then it could be that a large number of the pipeline passes are effectively a no-op on that IR.

If we do go down this proposed patch's route, we need to decide to what level do we want to make a pipeline completely representable as a string. Looking at PassBuilder.cpp, I don't think it's feasible to represent literally every possible pipeline as a string.
The legacy -debug-pass=Arguments decides to give up on handling pass params. It'll be a tradeoff of representing more vs more code changes/invasiveness. If we want to support pass params of any kind, each pass will have to map its params to a string (i.e. it can't be a static method). I'm not sure I really like adding more to PassInfoMixin.

I don't know what is the best approach here. Seems some sort of decision needs to be taken.

Currently we just have a mapping of names to a pass. This patch is basically trying to add a map going back the other way and also trying to keep it in sync with the PassRegistry.def map. It seems wrong to me... I never liked PassInstrumentationCallbacks::getPassNameForClassName(), I only added it to support --print-before/after.

Not sure that I understand this. There already is a mapping from ClassName to pass-name and I don't see how this patch is doing anything differently. Or is it so that this mapping should not exist in the first place.

If we're not going to use pass instrumentation, we shouldn't use PassInstrumentationCallbacks to get the pass name. We should do something similar where we gather up class -> pass names by including PassRegistry.def.

For sure, that could be refactored and the map could be placed elsewhere if we do agree that such a map should exist.

bjope added a comment.Aug 19 2021, 2:48 AM

In the end, when reducing a failure, we want to get down to a single pass pipeline, -passes='pass-name<params>'. Today, when -debug-pass-manager , -print-changed, etc. only prints the class name one need to open up the source code and try to map the class name to a pass name, and then also finding out which parameters that was used when it crashed.
Being able to print out the pipeline textually, after having applied all extensions etc, just before starting to run the pipeline (as this patch is aiming at) shouldn't be much different. And isn't printing the pipeline textually exactly what a tool such as opt-bisect wants to do when having finished up bisecting (regardless if the final reduced pipeline contains one or more passes).

I really think it would be useful to get the pass-name<params> printed automatically. Hence, there needs to be some kind of mapping from class name to pass name to exist somewhere, but we should also add support to print the parameters.
(Well, another solution to avoid the class name to pass name map would be to use class name all over the place, but that is quite a large change, specially while still having support for legacy PM.)

There are probably lots of different ways how to implement these things. Current situation with having information about a Pass in the PassRegistry.def and how to parse parameters in PassBuilder.cpp is a bit clumsy. All the information should probably be self-contained in the pass implementation somehow (I think this has been discussed earlier).
Not sure however if a potentially bad original design should stop us from adding the functionality that is missing. Hopefully someone will be eager to refactor things if the basic design of the new PM implementation is wrong, but I think it is even worse that the new PM implementation is lacking functionality. Not saying that I'm against refactoring first and adding new functionality afterwards, but sometimes it is hard for someone new to understand how to refactor things, while it might be easy to build upon the existing design.

bjope added a comment.Aug 20 2021, 6:12 AM

I started to look at some test directories, e.g. test/Transform/ADCE, and there are tests that for example run opt -adce -simplifycfg ..., and it is not obvious to me how to covert that to use -passes because I don't know which options to feed to simplifycfg.
If we manage to make this a bit more complete, also printing the options used, then I could run opt -adce -simplifycfg ... -print-pipeline or similar to get the equivalent textual pipeline description to use. So this might be helpful as a tool when converting lit tests to use -passes.

markus updated this revision to Diff 368097.Aug 23 2021, 7:21 AM

Updated to add framework support for printing pass parameters and implemented it for the SimplifyCFG pass.

$ opt -O3 -o /dev/null /dev/null
-passes='verify,annotation2metadata,forceattrs,inferattrs,function(lower-expect,simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts;>,sroa,early-cse,coro-early,callsite-splitting),openmp-opt,ipsccp,called-value-propagation,globalopt,function(mem2reg),deadargelim,function(instcombine,simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts;>),inliner-wra
pper,globalopt,globaldce,elim-avail-extern,rpo-function-attrs,function(float2int,lower-constant-intrinsics,loop(loop-rotate),loop-distribute,inject-tli-mappings,loop-vectorize,loop-load-elim,instcombine,simplifycfg<bonus-inst-threshold=1;forward-switch-cond;switch-to-lookup;no-keep-loops;hoist-common-insts;sink-common-insts;>,slp-vectorizer,vector-combine,instcombine,loop-unroll,transform-warning,instcombine,loop-mssa(licm),alignment-from-assumptions,loop-sink,instsimplify,div-rem-pairs,simplifycfg<bonus-
inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts;>,coro-cleanup),cg-profile,globaldce,constmerge,rel-lookup-table-converter,function(annotation-remarks),verify'
markus updated this revision to Diff 368854.Aug 26 2021, 5:30 AM
markus retitled this revision from [WIP][NPM] Print '-passes' compatible string for built pipeline. to [NPM] Print '-passes' compatible string for built pipeline..
markus edited the summary of this revision. (Show Details)

Cleaned up the patch, added command line option and removed [WIP] status.

markus added inline comments.Aug 26 2021, 5:31 AM
llvm/include/llvm/IR/PassManager.h
392

Ideally there should not be any passes that lack a mapping but for those that do it seems safer to print something so that it does not go unnoticed.

bjope added inline comments.Aug 26 2021, 6:37 AM
llvm/include/llvm/IR/PassManager.h
387

IMHO the getPassNameForClassName function is pretty trivial. Maybe you should just pass around a reference to the ClassToPassName StringMap instead. That will show that we do not really depend on or use the PassInstrumentationCallbacks class. We are only interested in the StringMap (and in the future that could be implemented somewhere else and not in the PassInstrumentationCallbacks class). Or what do you think about that?

llvm/lib/Passes/PassBuilder.cpp
446–447

Given the addition of the new PrintPipelinePasses option, wouldn't it be possible to use that one here now instead of true somehow.

A problem might be that PassBuilder is used by clang etc. So you'd need to move the PrintPipelinePasses option to PassBuilder.cpp (or maybe PrintPasses.cpp?).

llvm/tools/opt/NewPMDriver.cpp
479

I wonder if we perhaps want this to print to outs() instead, and then return true (instead of running the pipeline) here. Kind of similar to the -print-passes option.

markus updated this revision to Diff 368897.Aug 26 2021, 8:40 AM

Addressed review remarks.

markus marked 2 inline comments as done.Aug 26 2021, 8:42 AM
markus added inline comments.
llvm/include/llvm/IR/PassManager.h
387

Done or at least did something to the same effect.

markus updated this revision to Diff 369072.EditedAug 27 2021, 4:39 AM
markus edited the summary of this revision. (Show Details)

Added support for require<>, invalidate<> and repeat<>().

could you add some tests?
but don't check the exact output of something like -passes=default<O3>, we don't need more golden files to update when we update the pipeline. is it possible to get the output of one RUN line and use it in the next? to check if the output of -print-pipeline-passes -passes=default<O3> works.

we still have no guarantee that the generated output will exactly match the original pipeline. I think it's important to mention that in the description and in the flag.

markus updated this revision to Diff 369408.Aug 30 2021, 3:23 AM

Added lit-test, did some more cleanup.

Suggesting following commit message:

[NPM] Added opt option -print-pipeline-passes.

Added opt option -print-pipeline-passes to print a -passes compatible
string describing the built pass pipeline.

As an example:
$ opt -enable-new-pm=1 -adce -licm -simplifycfg -o /dev/null /dev/null -print-pipeline-passes
verify,function(adce),function(loop-mssa(licm)),function(simplifycfg<bonus-inst-threshold=1;no-forward-switch-cond;no-switch-to-lookup;keep-loops;no-hoist-common-insts;no-sink-common-insts>),verify,BitcodeWriterPass

At the moment this is best-effort only and there are some known
limitations:
- Not all passes accepting parameters will print their parameters
  (currently only implemented for simplifycfg).
- Some ClassName to pass-name mappings are not unique.
- Some ClassName to pass-name mappings are missing (e.g.
  BitcodeWriterPass).
markus added inline comments.Aug 30 2021, 3:27 AM
llvm/test/Other/new-pm-print-pipeline.ll
14 ↗(On Diff #369408)

It is tempting to add an additional test as follows as that would verify that an O3 pipeline is properly printed. May very well be that this only works on certain targets though so in that case the test needs to be moved to a target specific directory.

; RUN: opt -S /dev/null -passes=$(opt -O3 -S /dev/null -o /dev/null -print-pipeline-passes)
aeubanks added inline comments.Aug 30 2021, 10:35 AM
llvm/include/llvm/IR/PassManager.h
1309

does this not work with the default logic?

llvm/include/llvm/IR/PassManagerInternal.h
98

can this be part of printPipeline?

llvm/lib/Transforms/Scalar/LoopPassManager.cpp
50

function_ref is lighter weight than std::function

llvm/test/Other/new-pm-print-pipeline.ll
3 ↗(On Diff #369408)

-disable-output

3 ↗(On Diff #369408)

< %s seems nicer

4 ↗(On Diff #369408)

perhaps add -disable-verify to remove extra verifys?

markus updated this revision to Diff 369510.Aug 30 2021, 11:52 AM
markus marked 4 inline comments as done.

Addressed review remarks.

markus added inline comments.Aug 30 2021, 11:53 AM
llvm/include/llvm/IR/PassManager.h
1309

Apparently it does since PassRegistry.def contains

MODULE_PASS("invalidate<all>", InvalidateAllAnalysesPass())

So the implementation in PassInfoMixin handles it.
But this is different from the other invalidate<> though so IMHO it seems a bit nicer to be more explicit about it as there is a specific InvalidateAllAnalysesPass struct for it.

llvm/include/llvm/IR/PassManagerInternal.h
98

I don't think so. It needs to be separate so that it can be overridden separately in the Pass classes.

aeubanks added inline comments.Aug 31 2021, 10:27 PM
llvm/include/llvm/IR/PassManager.h
1309

to me having a separate InvalidateAllAnalysesPass is more of a reason to keep the default logic

llvm/include/llvm/IR/PassManagerInternal.h
98

I don't understand

why can't we for example have

void SimplifyCFGPass::printPipeline(raw_ostream &OS) {
  OS << name();
  OS << "<";
  OS << "bonus-inst-threshold=" << Options.BonusInstThreshold << ";";
  OS << (Options.ForwardSwitchCondToPhi ? "" : "no-") << "forward-switch-cond;";
  OS << (Options.ConvertSwitchToLookupTable ? "" : "no-")
     << "switch-to-lookup;";
  OS << (Options.NeedCanonicalLoop ? "" : "no-") << "keep-loops;";
  OS << (Options.HoistCommonInsts ? "" : "no-") << "hoist-common-insts;";
  OS << (Options.SinkCommonInsts ? "" : "no-") << "sink-common-insts";
  OS << ">";
}
llvm/test/Other/new-pm-print-pipeline.ll
3 ↗(On Diff #369408)

wait -disable-output shouldn't be necessary right? since we're not actually producing IR

markus added inline comments.Sep 1 2021, 12:12 AM
llvm/include/llvm/IR/PassManager.h
1309

Alright, I will adjust. In the end it does not really matter much though as long as we have regression tests to make sure that it works (which we do).

llvm/include/llvm/IR/PassManagerInternal.h
98

Ah, now I understand what you meant. Yes I agree that looks better.

llvm/test/Other/new-pm-print-pipeline.ll
3 ↗(On Diff #369408)

If we don't use it then either the print pass or the BitcodeWriterPass (that is lacking a pass-name) will be part of the pipeline. So keeping this option makes the tests a bit cleaner I think.

markus updated this revision to Diff 369872.Sep 1 2021, 12:14 AM

Addressed review remarks.

aeubanks accepted this revision.Sep 1 2021, 8:22 AM

I'd add one more test for loop-mssa, but other than that lgtm

This revision is now accepted and ready to land.Sep 1 2021, 8:22 AM
This revision was landed with ongoing or failed builds.Sep 1 2021, 11:30 PM
This revision was automatically updated to reflect the committed changes.

this doesn't work with the whole inliner cgscc pipeline (where most of the default simplification pipeline runs) because in ModuleInlinerWrapperPass::run we create a separate pass manager and run that
@mtrofin this sort of stuff is why I didn't particularly like the construction of a separate pass manager in ModuleInlinerWrapperPass

https://github.com/llvm/llvm-project/blob/124bcc1a139d744a71a82a89e90b5058573f2697/llvm/lib/Transforms/IPO/Inliner.cpp#L1049

this doesn't work with the whole inliner cgscc pipeline (where most of the default simplification pipeline runs) because in ModuleInlinerWrapperPass::run we create a separate pass manager and run that
@mtrofin this sort of stuff is why I didn't particularly like the construction of a separate pass manager in ModuleInlinerWrapperPass

https://github.com/llvm/llvm-project/blob/124bcc1a139d744a71a82a89e90b5058573f2697/llvm/lib/Transforms/IPO/Inliner.cpp#L1049

I think we need clarity wrt NPM and the scenarios that are pertinent to it, this included, because otherwise wrapping pipeline components into passes, maybe naively/subjectively, is attractive (FWIW, I could swear it was an intention of the design). So we either think that that's an accidental design artifact, and we document why it shouldn't be leveraged, or we offer guidelines (or extension points) for implementing cases like this 'print'. @aeubanks , can you help us with this?