This is an archive of the discontinued LLVM Phabricator instance.

[IPSCCP] Make sure Post Dominator Tree always is updated
AbandonedPublic

Authored by bjope on May 29 2023, 5:16 AM.

Details

Summary

Starting with commit 1b1232047e83b the IPSCCP pass may use
BlockFrequencyAnalysis. Since BlockFrequencyAnalysis is using
the PostDominatorTree (PDT) analysis a PDT might be calculated
for each function during IPSCCP, even if the analysis wasn't
cached before the pass.
Before that commit IPSCCP only made sure to update PDT
during transformations if the analysis was cached before the pass
(to make sure the the PDT was preserved rather than invalidated
in such scenarios).
The problem solved in this ticket is that commit 1b1232047e83b did
not change the criteria for when to update the PDT analysis. When
configuring the DomTreeUpdater it only provided the PDT if there
was a cached PDT analysis before IPSCCP started executing. Thus,
when the PDT was calculated as a side effect of running the
BlockFrequencyAnalysis, no DTU updates where made to that PDT, but
we still cached that (now invalid) PDT in the pass manager for future
passes.

Solution is a quick/simple fix to make sure that we calculate the
PDT before configuring the DTU. This to make sure we use the same PDT
when doing DTU updates as being used by the BlockFrequencyAnalysis.

A possible downside of this quick fix is that we might end up
calculating the PDT in many more scenarios (even when not using
the FunctionSpecializer+BlockFrequencyAnalysis). Maybe there are
ways to avoid that, but the goal with this patch is just to get rid
of crashes and potential miscompiles due to have a stale/invalid
PDT. An alternative would have been to revert 1b1232047e83b, but
there are already other patches being based on that commit so then
I would need to find all related patches and revert all of them.

Diff Detail

Event Timeline

bjope created this revision.May 29 2023, 5:16 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 29 2023, 5:16 AM
bjope requested review of this revision.May 29 2023, 5:16 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 29 2023, 5:16 AM
bjope updated this revision to Diff 526437.May 29 2023, 6:41 AM

Updated the new-pm pipeline tests that are impacted.

bjope retitled this revision from [ipsccp] Make sure Post Dominator Tree always is updated to [IPSCCP] Make sure Post Dominator Tree always is updated.May 29 2023, 6:46 AM

I have no clue if this messes up things (considering the compile-time measurements done for the related patches to IPSCCP/FunctionSpecialization). But I prefer if things are slow rather than faulty.

We've seen at least three different crashes/asserts in downstream testing due to this problem (probably with non-standard pipelines in fuzzy testing, so it is a bit annoying to leave this unfixed.

Looking at the impacted test cases it seems like PostDominatorTree isn't always available at beginning of IPSCCP even with default pipelines, so maybe the bug could manifest also with default pipelines(?).).

But I prefer if things are slow rather than faulty.

Both wrong, I would prefer revert of that commit and then build proper solution without any pressure.

nikic added a comment.May 29 2023, 7:37 AM

But I prefer if things are slow rather than faulty.

Both wrong, I would prefer revert of that commit and then build proper solution without any pressure.

Right, please revert the patch and anything based on it.

I checked, and (unnecessarily...) requiring the PDT here is not free: https://llvm-compile-time-tracker.com/compare.php?from=ab05d9134d18db34501985a01fbfc02609767587&to=bbb22133c4008a18559e48ebef215e7e6b5c82f0&stat=instructions:u

fhahn added a subscriber: fhahn.May 29 2023, 7:39 AM

Thanks for catching this subtle bug and tracking this down. Unconditionally computing the post-dominator tree here seems like it could be quite expensive. Like @xbolva00 said, it would probably be worth reverting the problematic commit, add the test case and recommit an updated patch that address the problem.

bjope added a comment.May 29 2023, 7:42 AM

But I prefer if things are slow rather than faulty.

Both wrong, I would prefer revert of that commit and then build proper solution without any pressure.

I wouldn't mind reverts. I had preferred if this was reverted directly when the problem was reported 5 days ago. Now I think there are a couple of patches that depend on this patch that needs to be reverted as well. I think it would be nice if @labrinea would deal with it somehow.

Meanwhile I've put this up to show a lit regression test, and to highlight the code that is wrong, and to show a possible workaround for anyone else who stumble upon the same problem.
Take it or leave it :-) (but if you leave it, please do the reverts or find a better solution)

bjope added a comment.May 29 2023, 7:55 AM

But I prefer if things are slow rather than faulty.

Both wrong, I would prefer revert of that commit and then build proper solution without any pressure.

Right, please revert the patch and anything based on it.

I checked, and (unnecessarily...) requiring the PDT here is not free: https://llvm-compile-time-tracker.com/compare.php?from=ab05d9134d18db34501985a01fbfc02609767587&to=bbb22133c4008a18559e48ebef215e7e6b5c82f0&stat=instructions:u

Thanks for checking. I kind of thought that the PDT would be calculated anyway given the use of the BFI (but I guess the BFI calculation is conditional and not done for every function).

bjope added a comment.EditedMay 29 2023, 8:38 AM

The commits to revert (without causing conflicts) seem to be:

  1. 0524534d5220da5ecb2c "[FuncSpec] Enable specialization of literal constants."
  2. ced90d1ff64a89a13479a "[FuncSpec] Improve the accuracy of the cost model."
  3. 13e3d4aa5a4e9062bb37 "[Pipeline] Don't run EarlyFPM in LTO post link"
  4. 9f992cc9350a7f "[SCCP] Fix -Wunused-lambda-capture"
  5. 1b1232047e83b69561fd "[FuncSpec] Replace LoopInfo with BlockFrequencyInfo."

I think commit (3) would be "collateral damage" as it seem to be unrelated (it was done by @aeubanks).

Apologies for the late reply, I was out of office. I have attempted to fix the problem here D151666. Please let me know if you have other suggestions.

bjope abandoned this revision.May 30 2023, 2:51 PM

The faulty code was reverted in https://reviews.llvm.org/rG96a14f388b1a3507e5ae97b0a21b7b785d99a52b
And an improved fix (including the reproducer from this patch) is being up for review in https://reviews.llvm.org/D151666

So this quick-fix workaround is not needed. Thanks!