[DWARF] Fix split dwarf debug inlining in mix mode situation.

[ayermolo]

Summary

With ThinLTO enabled we can have a function in a CU that doesn't have split
dwarf inlining enabled that gets inlined into CU that has it enabled.
When we try to lookup Abstract Scope Die for the inlined scope in
constructInlinedScopeDIE, it's not present and we assert.

Not 100% sure this is a correct place to fix it. Open to suggestions.

[ayermolo]

Also to add this also manifests if helper is build as monolithic.

[ayermolo]

@dblaikie Do you think you can review this, or someone else that you think would be better?

[dblaikie]

In D124892#3494463, @ayermolo wrote:

@dblaikie Do you think you can review this, or someone else that you think would be better?

This one's for me, for better and/or worse. I created this mess :)

[ayermolo]

In D124892#3494612, @dblaikie wrote:

In D124892#3494463, @ayermolo wrote:

@dblaikie Do you think you can review this, or someone else that you think would be better?

This one's for me, for better and/or worse. I created this mess :)

Thanks. I am not really sure if this is the right way to fix it. Personally seems hacky to me. I can explore other options if you have suggestions.

[dblaikie]

Yeah, mutating the DICompileUnit at this point seems a bad idea - it could end up inconsistent between things that happened before the mutation V after, etc.

Also the testing isn't suitable - LLVM tests can't depend on clang and the two IR files could be linked & the single IR result could be the test case? (it should just run llvm-mc - check other debug info tests). With the simplified/linked test, I can take a look and see if I can understand what's broken/how it might be suitable to fix it.

[ayermolo]

simplified test

[ayermolo]

In D124892#3494854, @dblaikie wrote:

Yeah, mutating the DICompileUnit at this point seems a bad idea - it could end up inconsistent between things that happened before the mutation V after, etc.

Also the testing isn't suitable - LLVM tests can't depend on clang and the two IR files could be linked & the single IR result could be the test case? (it should just run llvm-mc - check other debug info tests). With the simplified/linked test, I can take a look and see if I can understand what's broken/how it might be suitable to fix it.

Ah thanks!

[ayermolo]

@dblaikie Maybe you can point me in a direction where the fix could be better implemented? I can see what I can come up with.

[dblaikie]

With ThinLTO enabled we can have a function in a CU that doesn't have split dwarf inlining enabled that gets inlined into CU that has it enabled.
When we try to lookup Abstract Scope Die for the inlined scope in constructInlinedScopeDIE, it's not present and we assert.

So it seems there's probably some disconnect between the logic that adds abstract scope DIEs and the logic that looks them up? (one's using the source CU, one's using the destination CU?) - can you confirm that/which one's doing which?

& maybe them consistent would be good/suitable - I guess the right behavior is probably to treat the non-split-dwarf-inlining function as having no debug info for the purposes of the place it's inlined into?

[ayermolo]

In D124892#3529768, @dblaikie wrote:

With ThinLTO enabled we can have a function in a CU that doesn't have split dwarf inlining enabled that gets inlined into CU that has it enabled.
When we try to lookup Abstract Scope Die for the inlined scope in constructInlinedScopeDIE, it's not present and we assert.

So it seems there's probably some disconnect between the logic that adds abstract scope DIEs and the logic that looks them up? (one's using the source CU, one's using the destination CU?) - can you confirm that/which one's doing which?

& maybe them consistent would be good/suitable - I guess the right behavior is probably to treat the non-split-dwarf-inlining function as having no debug info for the purposes of the place it's inlined into?

So using this example.
When we are processing MachineFunction for main. We are processing SubProgram main, with CU in main.dwo. In constructAbstractSubprogramScopeDIE this is the SrcCU. It has debug inlining enabled.
What gets inlined in to it is Abstract Lexical Scope from helper.dwo that doesn't have debug inlining enabled.
Because of it we construct Abstract Subprogram Scope DIE only in SrcCU (Main DWO CU).

With my "fix" what happens is we end up creating CU for the helper dwo, Skeleton CU for it, and adding inline subprogram information to it for "func".
So when we construct subprogram scope die, when constructInlinedScopeDIE is invoked on scope for func (which comes from helper.cpp), we are able to find Inlined sub program and don't crash. Because we can find Origin DIE for inlined sub program.

Ideally I think we would like to have inlined sub program in main Skeleton CU. Otherwise it won't be complete. Although the way it is fixed now we end up with Skeleton CU for helper that is kind of broken (no DWO ID), and DWO CU.

Hopefully I am explaining it clearly. :)

[dblaikie]

In D124892#3532246, @ayermolo wrote:

In D124892#3529768, @dblaikie wrote:

With ThinLTO enabled we can have a function in a CU that doesn't have split dwarf inlining enabled that gets inlined into CU that has it enabled.
When we try to lookup Abstract Scope Die for the inlined scope in constructInlinedScopeDIE, it's not present and we assert.

So it seems there's probably some disconnect between the logic that adds abstract scope DIEs and the logic that looks them up? (one's using the source CU, one's using the destination CU?) - can you confirm that/which one's doing which?

& maybe them consistent would be good/suitable - I guess the right behavior is probably to treat the non-split-dwarf-inlining function as having no debug info for the purposes of the place it's inlined into?

So using this example.
When we are processing MachineFunction for main. We are processing SubProgram main, with CU in main.dwo. In constructAbstractSubprogramScopeDIE this is the SrcCU. It has debug inlining enabled.
What gets inlined in to it is Abstract Lexical Scope from helper.dwo that doesn't have debug inlining enabled.
Because of it we construct Abstract Subprogram Scope DIE only in SrcCU (Main DWO CU).

With my "fix" what happens is we end up creating CU for the helper dwo, Skeleton CU for it, and adding inline subprogram information to it for "func".
So when we construct subprogram scope die, when constructInlinedScopeDIE is invoked on scope for func (which comes from helper.cpp), we are able to find Inlined sub program and don't crash. Because we can find Origin DIE for inlined sub program.

Ideally I think we would like to have inlined sub program in main Skeleton CU. Otherwise it won't be complete.

I think that's probably where I'm suggesting differently: If someone didn't build SrcCU with split-dwarf-inlining, then even if that code gets inlined into MainCU that /does/ have split-dwarf-inlining, it would be OK/consistent /not/ to describe the inlined function from SrcCU. I think that'd be correctly respecting the level of detail the user asked for - no symbolizing for functions in (or from) SrcCU.

Could you implement that? By checking when we go to construct the inlined scope DIE if the subprogram comes from a CU without split-dwarf-inlining, and if so, skip creating the split dwarf inlined description?

[ayermolo]

In D124892#3538807, @dblaikie wrote:

In D124892#3532246, @ayermolo wrote:

In D124892#3529768, @dblaikie wrote:

With ThinLTO enabled we can have a function in a CU that doesn't have split dwarf inlining enabled that gets inlined into CU that has it enabled.
When we try to lookup Abstract Scope Die for the inlined scope in constructInlinedScopeDIE, it's not present and we assert.

So it seems there's probably some disconnect between the logic that adds abstract scope DIEs and the logic that looks them up? (one's using the source CU, one's using the destination CU?) - can you confirm that/which one's doing which?

& maybe them consistent would be good/suitable - I guess the right behavior is probably to treat the non-split-dwarf-inlining function as having no debug info for the purposes of the place it's inlined into?

So using this example.
When we are processing MachineFunction for main. We are processing SubProgram main, with CU in main.dwo. In constructAbstractSubprogramScopeDIE this is the SrcCU. It has debug inlining enabled.
What gets inlined in to it is Abstract Lexical Scope from helper.dwo that doesn't have debug inlining enabled.
Because of it we construct Abstract Subprogram Scope DIE only in SrcCU (Main DWO CU).

With my "fix" what happens is we end up creating CU for the helper dwo, Skeleton CU for it, and adding inline subprogram information to it for "func".
So when we construct subprogram scope die, when constructInlinedScopeDIE is invoked on scope for func (which comes from helper.cpp), we are able to find Inlined sub program and don't crash. Because we can find Origin DIE for inlined sub program.

Ideally I think we would like to have inlined sub program in main Skeleton CU. Otherwise it won't be complete.

I think that's probably where I'm suggesting differently: If someone didn't build SrcCU with split-dwarf-inlining, then even if that code gets inlined into MainCU that /does/ have split-dwarf-inlining, it would be OK/consistent /not/ to describe the inlined function from SrcCU. I think that'd be correctly respecting the level of detail the user asked for - no symbolizing for functions in (or from) SrcCU.

Could you implement that? By checking when we go to construct the inlined scope DIE if the subprogram comes from a CU without split-dwarf-inlining, and if so, skip creating the split dwarf inlined description?

Sure. Let me give it a shot next week.

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp

 void DwarfDebug::constructAbstractSubprogramScopeDIE(DwarfCompileUnit &SrcCU,
                                                      LexicalScope *Scope) {
   assert(Scope && Scope->getScopeNode());
   assert(Scope->isAbstractScope());
   assert(!Scope->getInlinedAt());
 
   auto *SP = cast<DISubprogram>(Scope->getScopeNode());
 
+  // Handling a case with LTO where function is inlined from CU which doesn't
+  // have SplitDebugInlining enabled into one that does.
+  if (SrcCU.getCUNode()->getSplitDebugInlining())
+    SP->getUnit()->setSplitDebugInlining(true);
+
   // Find the subprogram's DwarfCompileUnit in the SPMap in case the subprogram
   // was inlined from another compile unit.
   if (useSplitDwarf() && !shareAcrossDWOCUs() && !SP->getUnit()->getSplitDebugInlining())
     // Avoid building the original CU if it won't be used
     SrcCU.constructAbstractSubprogramScopeDIE(Scope);
   else {
     auto &CU = getOrCreateDwarfCompileUnit(SP->getUnit());
     if (auto *SkelCU = CU.getSkeleton()) {

llvm/test/DebugInfo/X86/mixDebugInliing.test

+; RUN: llc -mtriple=x86_64-linux -split-dwarf-file=foo.dwo -filetype=obj -o %t < %s
+; RUN: llvm-dwarfdump --debug-info --show-form --verbose %t | FileCheck -check-prefix=CHECK %s
+
+; CHECK:        DW_TAG_compile_unit
+; CHECK-NEXT:       DW_AT_stmt_list
+; CHECK-NEXT:       DW_AT_comp_dir
+; CHECK-NEXT:       DW_AT_GNU_pubnames
+; CHECK-NEXT:       DW_AT_GNU_dwo_name
+; CHECK-NEXT:       DW_AT_GNU_dwo_id
+; CHECK-NEXT:       DW_AT_low_pc
+; CHECK-NEXT:       DW_AT_high_pc
+; CHECK-NEXT:       DW_AT_GNU_addr_base
+; CHECK-EMPTY:
+; CHECK-NEXT:   DW_TAG_subprogram
+; CHECK-NEXT:       DW_AT_low_pc
+; CHECK-NEXT:       DW_AT_high_pc
+; CHECK-NEXT:       DW_AT_name
+; CHECK-EMPTY:
+; CHECK-NEXT:   DW_TAG_inlined_subroutine
+; CHECK-NEXT:       DW_AT_abstract_origin
+; CHECK-NEXT:       DW_AT_low_pc
+; CHECK-NEXT:       DW_AT_high_pc
+; CHECK-NEXT:       DW_AT_call_file
+; CHECK-NEXT:       DW_AT_call_line
+; CHECK-NEXT:       DW_AT_call_column
+
+; ModuleID = 'mainhelper.ll'
+source_filename = "llvm-link"
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; Function Attrs: noinline norecurse optnone uwtable mustprogress
+define dso_local i32 @main(i32 %argc, i8** %argv) #0 !dbg !9 {
+entry:
+  %x.addr.i = alloca i32, align 4
+  %retval = alloca i32, align 4
+  %argc.addr = alloca i32, align 4
+  %argv.addr = alloca i8**, align 8
+  store i32 0, i32* %retval, align 4
+  store i32 %argc, i32* %argc.addr, align 4
+  call void @llvm.dbg.declare(metadata i32* %argc.addr, metadata !16, metadata !DIExpression()), !dbg !17
+  store i8** %argv, i8*** %argv.addr, align 8
+  call void @llvm.dbg.declare(metadata i8*** %argv.addr, metadata !18, metadata !DIExpression()), !dbg !19
+  %0 = load i32, i32* %argc.addr, align 4, !dbg !20
+  store i32 %0, i32* %x.addr.i, align 4
+  call void @llvm.dbg.declare(metadata i32* %x.addr.i, metadata !21, metadata !DIExpression()), !dbg !25
+  %1 = load i32, i32* %x.addr.i, align 4, !dbg !27
+  %mul.i = mul nsw i32 %1, 2, !dbg !28
+  ret i32 %mul.i, !dbg !29
+}
+
+; Function Attrs: nofree nosync nounwind readnone speculatable willreturn
+declare void @llvm.dbg.declare(metadata, metadata, metadata) #1
+
+; Function Attrs: alwaysinline nounwind uwtable mustprogress
+define dso_local i32 @_Z4funci(i32 %x) #2 !dbg !22 {
+entry:
+  %x.addr = alloca i32, align 4
+  store i32 %x, i32* %x.addr, align 4
+  call void @llvm.dbg.declare(metadata i32* %x.addr, metadata !21, metadata !DIExpression()), !dbg !30
+  %0 = load i32, i32* %x.addr, align 4, !dbg !31
+  %mul = mul nsw i32 %0, 2, !dbg !32
+  ret i32 %mul, !dbg !33
+}
+
+attributes #0 = { noinline norecurse optnone uwtable mustprogress "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { nofree nosync nounwind readnone speculatable willreturn }
+attributes #2 = { alwaysinline nounwind uwtable mustprogress "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" "unsafe-fp-math"="false" "use-soft-float"="false" }
+
+!llvm.dbg.cu = !{!0, !3}
+!llvm.ident = !{!5, !5}
+!llvm.module.flags = !{!6, !7, !8}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, producer: "clang version 12.0.1", isOptimized: false, runtimeVersion: 0, splitDebugFilename: "main.dwo", emissionKind: FullDebug, enums: !2, nameTableKind: GNU)
+!1 = !DIFile(filename: "main.cpp", directory: "bug2")
+!2 = !{}
+!3 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !4, producer: "clang version 12.0.1", isOptimized: false, runtimeVersion: 0, splitDebugFilename: "helper.dwo", emissionKind: FullDebug, enums: !2, splitDebugInlining: false, nameTableKind: GNU)
+!4 = !DIFile(filename: "helper.cpp", directory: "bug2")
+!5 = !{!"clang version 12.0.1"}
+!6 = !{i32 7, !"Dwarf Version", i32 4}
+!7 = !{i32 2, !"Debug Info Version", i32 3}
+!8 = !{i32 1, !"wchar_size", i32 4}
+!9 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 2, type: !10, scopeLine: 2, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!10 = !DISubroutineType(types: !11)
+!11 = !{!12, !12, !13}
+!12 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!13 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !14, size: 64)
+!14 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !15, size: 64)
+!15 = !DIBasicType(name: "char", size: 8, encoding: DW_ATE_signed_char)
+!16 = !DILocalVariable(name: "argc", arg: 1, scope: !9, file: !1, line: 2, type: !12)
+!17 = !DILocation(line: 2, column: 14, scope: !9)
+!18 = !DILocalVariable(name: "argv", arg: 2, scope: !9, file: !1, line: 2, type: !13)
+!19 = !DILocation(line: 2, column: 26, scope: !9)
+!20 = !DILocation(line: 3, column: 15, scope: !9)
+!21 = !DILocalVariable(name: "x", arg: 1, scope: !22, file: !4, line: 1, type: !12)
+!22 = distinct !DISubprogram(name: "func", linkageName: "_Z4funci", scope: !4, file: !4, line: 1, type: !23, scopeLine: 1, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !3, retainedNodes: !2)
+!23 = !DISubroutineType(types: !24)
+!24 = !{!12, !12}
+!25 = !DILocation(line: 1, column: 45, scope: !22, inlinedAt: !26)
+!26 = distinct !DILocation(line: 3, column: 10, scope: !9)
+!27 = !DILocation(line: 2, column: 11, scope: !22, inlinedAt: !26)
+!28 = !DILocation(line: 2, column: 13, scope: !22, inlinedAt: !26)
+!29 = !DILocation(line: 3, column: 3, scope: !9)
+!30 = !DILocation(line: 1, column: 45, scope: !22)
+!31 = !DILocation(line: 2, column: 11, scope: !22)
+!32 = !DILocation(line: 2, column: 13, scope: !22)
+!33 = !DILocation(line: 2, column: 4, scope: !22)