This patch adds a depth limit for the recursive alias analysis queries in BasicAA. Unfortunately, this is pretty tricky, because we need to think about how this interacts with BatchAA. Just doing a naive cutoff means that BatchAA can produce worse or better results depending on the order of queries, which is something we want to avoid. For this reason, this patch tracks the starting depth and maximum depth for cache entries, to determine whether a cached result can be used at a given depth.
For the depth limit of 8 used here, we get the following stats differences at O3:
aa.NumMayAlias | 5010256 | 5010623 aa.NumNoAlias | 27200820 | 27200306
This is also a minor compile-time regression: https://llvm-compile-time-tracker.com/compare.php?from=20cb6c7cebb5a3566640a9bf0da4729993ce6020&to=5e9effe146ed5d2a3934484db2493ffd66b0b5df&stat=instructions You can also see that with ThinLTO mafft regresses (0.87% codesize increase).
This addresses https://bugs.llvm.org/show_bug.cgi?id=49151, where BasicAA recurses enough to overflow the stack. Possibly D92401 could be reapplied, as it ran into exponential complexity in some cases.