[LiveDebugValues] Add cutoffs to avoid pathological behavior

[vsk]

Summary

We received a report of LiveDebugValues consuming 25GB+ of RAM when
compiling code generated by Unity's IL2CPP scripting backend.

There's an initial 5GB spike due to repeatedly copying cached lists of
MachineBasicBlocks within the UserValueScopes members of VarLocs.

But the larger scaling issue arises due to the fact that prior to range
extension, there are 81K basic blocks and 156K DBG_VALUEs: given enough
memory, LiveDebugValues would insert 101 million MIs (I counted this by
incrementing a counter inside of VarLoc::BuildDbgValue).

It seems like LiveDebugValues would have to be rearchitected to support
this kind of input (we'd need some new represntation for DBG_VALUEs that
get inserted into ~every block via flushPendingLocs). OTOH, large globs
of auto-generated code are typically not debugged interactively.

So: add cutoffs to disable range extension when the input is too big. I
chose the cutoffs experimentally, erring on the conservative side. When
compiling a large collection of Apple software, range extension never
got disabled.

rdar://63418929

[djtodoro]

I think this is reasonable. I think the implementation of the Value Tracking within GCC uses a magic value where it applies a similar cutoff if the number of BBs is higher than that.

[jmorse]

LGTM, definitely better to bail out than to try something impossible.

llvm/lib/CodeGen/LiveDebugValues.cpp

1769

Iterating over all instructions again is probably unfortunate for performance -- it could be counted on line 1744 of this patch?

On the other hand, given the exceptional circumstances that this will trigger in, that's not really a concern.

[vsk]

Thanks for the reviews!

llvm/lib/CodeGen/LiveDebugValues.cpp

1769

It seems a bit cleaner to separate the concerns. For inputs where the extra scan would be a performance issue, the pass is likely to bail out anyway.

llvm/lib/CodeGen/LiveDebugValues.cpp

 
 using namespace llvm;
 
 #define DEBUG_TYPE "livedebugvalues"
 
 STATISTIC(NumInserted, "Number of DBG_VALUE instructions inserted");
 STATISTIC(NumRemoved, "Number of DBG_VALUE instructions removed");
 
+// Options to prevent pathological compile-time behavior. If InputBBLimit and
+// InputDbgValueLimit are both exceeded, range extension is disabled.
+static cl::opt<unsigned> InputBBLimit(
+    "livedebugvalues-input-bb-limit",
+    cl::desc("Maximum input basic blocks before DBG_VALUE limit applies"),
+    cl::init(10000), cl::Hidden);
+static cl::opt<unsigned> InputDbgValueLimit(
+    "livedebugvalues-input-dbg-value-limit",
+    cl::desc(
+        "Maximum input DBG_VALUE insts supported by debug range extension"),
+    cl::init(50000), cl::Hidden);
+
 // If @MI is a DBG_VALUE with debug value described by a defined
 // register, returns the number of this register. In the other case, returns 0.
 static Register isDbgValueDescribedByReg(const MachineInstr &MI) {
   assert(MI.isDebugValue() && "expected a DBG_VALUE");
   assert(MI.getNumOperands() == 4 && "malformed DBG_VALUE");
   // If location of variable is described using a register (directly
   // or indirectly), this register is always a first operand.
   return MI.getOperand(0).isReg() ? MI.getOperand(0).getReg() : Register();
bool LiveDebugValues::ExtendRanges(MachineFunction &MF) {
   ReversePostOrderTraversal<MachineFunction *> RPOT(&MF);
   unsigned int RPONumber = 0;
   for (auto RI = RPOT.begin(), RE = RPOT.end(); RI != RE; ++RI) {
     OrderToBB[RPONumber] = *RI;
     BBToOrder[*RI] = RPONumber;
     Worklist.push(RPONumber);
     ++RPONumber;
   }
+
+  if (RPONumber > InputBBLimit) {
+    unsigned NumInputDbgValues = 0;
+    for (auto &MBB : MF)
+      for (auto &MI : MBB)
+        if (MI.isDebugValue())
+          ++NumInputDbgValues;
+    if (NumInputDbgValues > InputDbgValueLimit) {
+      LLVM_DEBUG(dbgs() << "Disabling LiveDebugValues: " << MF.getName()
+                        << " has " << RPONumber << " basic blocks and "
+                        << NumInputDbgValues
+                        << " input DBG_VALUEs, exceeding limits.\n");
+      return false;
+    }
+  }
+
   // This is a standard "union of predecessor outs" dataflow problem.
   // To solve it, we perform join() and process() using the two worklist method
   // until the ranges converge.
   // Ranges have converged when both worklists are empty.
   SmallPtrSet<const MachineBasicBlock *, 16> Visited;
   while (!Worklist.empty() || !Pending.empty()) {
     // We track what is on the pending worklist to avoid inserting the same
     // thing twice.  We could avoid this with a custom priority queue, but this

llvm/test/DebugInfo/MIR/X86/live-debug-values-cutoffs.mir

+# Test cutoffs for livedebugvalues debug range extension.
+# Disable LDV if the input-bb-limit AND the input-dbg-value-limit are both exceeded.
+
+# RUN: llc %s -o - -run-pass=livedebugvalues -mtriple=x86_64-unknown-unknown \
+# RUN:   -livedebugvalues-input-bb-limit=1 \
+# RUN:   -livedebugvalues-input-dbg-value-limit=1 \
+# RUN:   | FileCheck %s -check-prefix=LDV-DISABLED
+
+# RUN: llc %s -o - -run-pass=livedebugvalues -mtriple=x86_64-unknown-unknown \
+# RUN:   -livedebugvalues-input-bb-limit=1 \
+# RUN:   -livedebugvalues-input-dbg-value-limit=10 \
+# RUN:   | FileCheck %s -check-prefix=LDV-ENABLED
+
+# RUN: llc %s -o - -run-pass=livedebugvalues -mtriple=x86_64-unknown-unknown \
+# RUN:   -livedebugvalues-input-bb-limit=10 \
+# RUN:   -livedebugvalues-input-dbg-value-limit=1 \
+# RUN:   | FileCheck %s -check-prefix=LDV-ENABLED
+
+# RUN: llc %s -o - -run-pass=livedebugvalues -mtriple=x86_64-unknown-unknown \
+# RUN:   -livedebugvalues-input-bb-limit=10 \
+# RUN:   -livedebugvalues-input-dbg-value-limit=10 \
+# RUN:   | FileCheck %s -check-prefix=LDV-ENABLED
+
+# LDV-DISABLED-LABEL: bb.1.exit
+# LDV-DISABLED-NEXT: $edi = MOV32rm
+
+# LDV-ENABLED-LABEL: bb.1.exit
+# LDV-ENABLED-NEXT: DBG_VALUE $rsp, 0, {{.*}}, !DIExpression(DW_OP_plus_uconst, 4)
+# LDV-ENABLED-NEXT: $edi = MOV32rm
+
+--- |
+  target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+
+  declare i32 @use(i32)
+
+  define i32 @foo(i32 %x) !dbg !6 {
+  entry:
+    %y = add i32 %x, %x, !dbg !12
+    call void @llvm.dbg.value(metadata i32 %y, metadata !9, metadata !DIExpression()), !dbg !12
+    br label %exit, !dbg !13
+
+  exit:                                             ; preds = %entry
+    %z = call i32 @use(i32 %y), !dbg !14
+    call void @llvm.dbg.value(metadata i32 %z, metadata !11, metadata !DIExpression()), !dbg !14
+    ret i32 %z, !dbg !15
+  }
+
+  declare void @llvm.dbg.value(metadata, metadata, metadata)
+
+  !llvm.dbg.cu = !{!0}
+  !llvm.debugify = !{!3, !4}
+  !llvm.module.flags = !{!5}
+
+  !0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "debugify", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
+  !1 = !DIFile(filename: "/tmp/t.ll", directory: "/")
+  !2 = !{}
+  !3 = !{i32 4}
+  !4 = !{i32 2}
+  !5 = !{i32 2, !"Debug Info Version", i32 3}
+  !6 = distinct !DISubprogram(name: "foo", linkageName: "foo", scope: null, file: !1, line: 1, type: !7, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !8)
+  !7 = !DISubroutineType(types: !2)
+  !8 = !{!9, !11}
+  !9 = !DILocalVariable(name: "1", scope: !6, file: !1, line: 1, type: !10)
+  !10 = !DIBasicType(name: "ty32", size: 32, encoding: DW_ATE_unsigned)
+  !11 = !DILocalVariable(name: "2", scope: !6, file: !1, line: 3, type: !10)
+  !12 = !DILocation(line: 1, column: 1, scope: !6)
+  !13 = !DILocation(line: 2, column: 1, scope: !6)
+  !14 = !DILocation(line: 3, column: 1, scope: !6)
+  !15 = !DILocation(line: 4, column: 1, scope: !6)
+
+...
+---
+name:            foo
+liveins:
+  - { reg: '$edi', virtual-reg: '' }
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -12, size: 4, alignment: 4,
+      stack-id: default, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+body:             |
+  bb.0.entry:
+    successors: %bb.1(0x80000000)
+    liveins: $edi
+
+    frame-setup PUSH64r undef $rax, implicit-def $rsp, implicit $rsp
+    CFI_INSTRUCTION def_cfa_offset 16
+    renamable $edi = ADD32rr renamable $edi, killed renamable $edi, implicit-def $eflags, debug-location !12
+    DBG_VALUE renamable $edi, $noreg, !9, !DIExpression(), debug-location !12
+    MOV32mr $rsp, 1, $noreg, 4, $noreg, killed $edi :: (store 4 into %stack.0)
+    DBG_VALUE $rsp, 0, !9, !DIExpression(DW_OP_plus_uconst, 4), debug-location !12
+
+  bb.1.exit:
+    $edi = MOV32rm $rsp, 1, $noreg, 4, $noreg :: (load 4 from %stack.0)
+    CALL64pcrel32 @use, csr_64, implicit $rsp, implicit $ssp, implicit killed $edi, implicit-def $eax, debug-location !14
+    DBG_VALUE renamable $eax, $noreg, !11, !DIExpression(), debug-location !14
+    $rcx = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !15
+    RETQ implicit killed $eax, debug-location !15
+
+...