This is an archive of the discontinued LLVM Phabricator instance.

Prevent explosion of debug intrinsics during jump threading
ClosedPublic

Authored by StephenTozer on Jan 20 2020, 9:24 AM.

Details

Summary

This patch is a fix following the revert of 72ce759 (https://reviews.llvm.org/rG72ce759928e6dfee6a9efa310b966c19722352ba) and fixes the failure that it caused.

The above patch failed on the Thread Sanitizer buildbot with an out of memory error. After an investigation, the cause was identified as an explosion in debug intrinsics while running the Jump Threading pass on ModuleMap.ll. The above patched prevented debug intrinsics from being dropped when their Basic Block was deleted due to being "empty". In this case, one of the functions in ModuleMap.ll had (after many optimization passes) a very large number of debug intrinsics representing a set of repeatedly inlined variables. Previously the vast majority of these were silently dropped during Jump Threading when their blocks were deleted, but as of the above patch they survived for longer, causing a large increase in the number of debug intrinsics. These intrinsics were then repeatedly cloned by the Jump Threading pass as edges were threaded, multiplying the intrinsic count further. The memory consumed by this process spiralled out of control, crashing the buildbot that uses TSan (which has an estimated 5-10x memory overhead compared to non-sanitized builds).

In order to rein in the number of debug intrinsics, I have added RemoveRedundantDbgInstrs to the Jump Threading pass. This recently added function (https://reviews.llvm.org/D71478) processes a Basic Block and removes any debug intrinsics that are "redundant", i.e. they are either identical to a previous intrinsic for the same source variable in the same block, or immediately overwritten by another intrinsic with no intermediate instructions. As many intrinsics for the same variable end up propagated into the same blocks during this case, the removal of redundant intrinsics from each block as they are processed is effective are reducing the number of intrinsics from exploding in the same manner.

This seems to bring the number of intrinsics for this case down to a manageable level in practice, and should be sufficient to unblock the above patch for remerging. It does seem likely to me that there are other passes for which this function would also be appropriate; this is probably worth looking into after the immediate problem is solved.

Diff Detail

Event Timeline

StephenTozer created this revision.Jan 20 2020, 9:24 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 20 2020, 9:24 AM

Thanks for working on this! Please add a test case.

Orlando added inline comments.
llvm/lib/Transforms/Scalar/JumpThreading.cpp
21

nit: I think RemoveRedundantDbgInsts should moved above the comment it currently follows.

Add test, address nit comment. Note that some of the current test checks should pass even without this patch if commit 636c93 is not reverted, as some of the redundant debug info would be dropped.

djtodoro added inline comments.Jan 23 2020, 2:29 AM
llvm/lib/Transforms/Scalar/JumpThreading.cpp
409

*threading*

Fix spelling error.

aprantl accepted this revision.Jan 24 2020, 2:33 PM
This revision is now accepted and ready to land.Jan 24 2020, 2:33 PM
djtodoro accepted this revision.Feb 12 2020, 12:13 AM
This revision was automatically updated to reflect the committed changes.