Relevant discussion can be found at: https://lists.llvm.org/pipermail/llvm-dev/2021-January/148197.html
In the existing design, An SCC that contains a coroutine will go through the folloing passes:
Inliner -> CoroSplitPass (fake) -> FunctionSimplificationPipeline -> Inliner -> CoroSplitPass (real) -> FunctionSimplificationPipeline
The first CoroSplitPass doesn't do anything other than putting the SCC back to the queue so that the entire pipeline can repeat.
As you can see, we run Inliner twice on the SCC consecutively without doing any real split, which is unnecessary and likely unintended.
What we really wanted is this:
Inliner -> FunctionSimplificationPipeline -> CoroSplitPass -> Inliner -> FunctionSimplificationPipeline
Hence the way we do it here is to move CoroSplitPass to the end of the CGSCC pipeline, make it once for real, and then insert the newly generated SCCs back to the pipeline so that they can be optimized again.
From here we can then reason about whether we actually want to run it a second time (maybe add an option to turn it off if the cost is over benefit).
This approach also conforms to how the new pass manager works instead of relying on an adhoc post split cleanup, making it ready for full switch to new pass manager eventually.
A lot of tests need to be updated because we now no longer clean things up in O0 after CoroSplit.
I will clean up the tests latter but for now this is RFC to see if people have any objects on doing this.