[dsymutil] Emit valid debug locations when no symbol flags are set

[fjricci]

Summary

swiftc emits symbols without flags set, which led dsymutil to ignore
them when searching for global symbols, causing dwarf location data
to be omitted. Xcode's dsymutil handles this case correctly, and emits
valid location data. Add this functionality to llvm-dsymutil by
allowing parsing of symbols with no flags set.

[fjricci]

I'm going to try to clean up the IR a bit

[fjricci]

Prune some unneeded IR and debug info

[aprantl]

tools/dsymutil/MachODebugMapParser.cpp

485–486

Note that

https://opensource.apple.com/source/clang/clang-800.0.42.1/src/tools/dsymutil/MachODebugMapParser.cpp.auto.html

uses SymbolRef::SF_Common instead.

[fjricci]

tools/dsymutil/MachODebugMapParser.cpp

485–486

Ah, I didn't realize that the apple fork was open source as well. Are there plans to merge it back into llvm? (probably applies to https://reviews.llvm.org/D38504 as well)

[fjricci]

tools/dsymutil/MachODebugMapParser.cpp

485–486

If there are plans to eventually merge the two, these two diffs are just going to cause the forks to diverge and make merging harder (unless I merge them in as copies from what's in the apple version)

[aprantl]

Yes, we are in the process of contributing all changes where this makes sense for back into the llvm.org repository. I wouldn't worry too much about introducing merge conflicts (we can handle that), but we shouldn't change the behavior of the tool (unless you have found a bug in the shipping dsymutil, of course).

[JDevlieghere]

Looks good with Adrian's comments addressed.

[fjricci]

Maintain same behavior as apple's dsymutil

[fjricci]

Move to x86 test dir

[fjricci]

Use precompiled binary with IR as a comment to allow testing on linux

[fjricci]

I'm running into some issues with using a precompiled binary + object file here. For some reason, the debug info contains an absolute path to the object file instead of a relative path, so dsymutil will fail because it's looking for a path that doesn't exist (ie a path on my local machine). This is true even if I use -fdebug-compilation-dir (it updates the debug info in the IR, but doesn't change where dsymutil looks for the object file). I also tried -add_ast_path, which works as a hacky workaround because it will search the relative path in addition to the absolute path, but that seems like a bad solution.

Interestingly, I tried to reproduce the test object files created in one of the other tests (X86/alias.test), and when I regenerate the object files locally, they switch to absolute paths as well and the test fails.

One way around this would be to go back to compiling/linking the IR file as part of the test, but this requires adding REQUIRES: darwin, for linking to work.

Any thoughts?

[aprantl]

Let me know if I misunderstood your question. When creating binary testcases for dsymutil, I usually create them directly in / so the absolute paths don't contain any local directories.

[fjricci]

Oh interesting. I tried using PWD=/, but I didn't try actually cd-ing into that directory. I'll try that - thanks!

test/tools/dsymutil/Inputs/swift-dwarf-loc.ll

+; ModuleID = '-'
+source_filename = "-"
+target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-macosx10.12"
+
+%TSi = type <{ i64 }>
+
+@_var = hidden global %TSi zeroinitializer, align 8, !dbg !0
+
+!llvm.dbg.cu = !{!8}
+!llvm.module.flags = !{!11, !12}
+
+!0 = !DIGlobalVariableExpression(var: !1, expr: !DIExpression())
+!1 = distinct !DIGlobalVariable(name: "x", linkageName: "_var", scope: !2, file: !3, line: 1, type: !4, isLocal: false, isDefinition: true)
+!2 = !DIModule(scope: null, name: "main")
+!3 = !DIFile(filename: "<stdin>", directory: "")
+!4 = !DICompositeType(tag: DW_TAG_structure_type, name: "Int", scope: !6, file: !5, size: 64, elements: !7, runtimeLang: DW_LANG_Swift, identifier: "_T0SiD")
+!5 = !DIFile(filename: "foo", directory: "/tmp")
+!6 = !DIModule(scope: null, name: "foo", includePath: "")
+!7 = !{}
+!8 = distinct !DICompileUnit(language: DW_LANG_Swift, file: !9, producer: "swiftc", isOptimized: false, flags: "", runtimeVersion: 4, emissionKind: FullDebug, enums: !7, globals: !10, imports: null)
+!9 = !DIFile(filename: "/tmp", directory: "")
+!10 = !{!0}
+!11 = !{i32 2, !"Dwarf Version", i32 4}
+!12 = !{i32 2, !"Debug Info Version", i32 3}

test/tools/dsymutil/swift-dwarf-loc.test

+RUN: llc -filetype=obj %p/Inputs/swift-dwarf-loc.ll -mtriple x86_64-apple-darwin -o %T/swift-dwarf-loc.o
+RUN: ld -dylib %T/swift-dwarf-loc.o -arch x86_64 -lSystem -macosx_version_min 10.9.0 -o %T/swift-dwarf-loc
+RUN: llvm-dsymutil %T/swift-dwarf-loc -no-output -verbose | FileCheck %s
+
+This test checks that dsymutil generates a valid dwarf location for a symbol with no flags set.
+
+CHECK: __var,
+CHECK-NOT: __var,{{.*}}binAddr: 0x0000000000000000
+CHECK-NOT: __var{{.*}} => 0000000000000000

tools/dsymutil/MachODebugMapParser.cpp

for (const auto &Sym : MainBinary.symbols()) {
     SymbolRef::Type Type = *TypeOrErr;
     // Skip undefined and STAB entries.
     if ((Type == SymbolRef::ST_Debug) || (Type == SymbolRef::ST_Unknown))
       continue;
     // The only symbols of interest are the global variables. These
     // are the only ones that need to be queried because the address
     // of common data won't be described in the debug map. All other
     // addresses should be fetched for the debug map.
-    if (!(Sym.getFlags() & SymbolRef::SF_Global))
+    uint8_t SymType = MainBinary.getSymbolTableEntry(Sym.getRawDataRefImpl()).n_type;
+    if (!(SymType & (MachO::N_EXT | MachO::N_PEXT)))
       continue;
     Expected<section_iterator> SectionOrErr = Sym.getSection();
     if (!SectionOrErr) {
       // TODO: Actually report errors helpfully.
       consumeError(SectionOrErr.takeError());
       continue;
     }
     Section = *SectionOrErr;