This is an archive of the discontinued LLVM Phabricator instance.

Fix a double debug info size counting in top level stats for "statistics dump".
ClosedPublic

Authored by clayborg on Feb 9 2022, 7:12 PM.

Details

Summary

This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list.

So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files.

Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress.

For a module with a dSYM file, we can see the "symbolFilePath" is included:

"modules": [
  {
    "debugInfoByteSize": 1070,
    "debugInfoIndexLoadedFromCache": false,
    "debugInfoIndexSavedToCache": false,
    "debugInfoIndexTime": 0,
    "debugInfoParseTime": 0,
    "identifier": 4873280600,
    "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out",
    "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out",
    "symbolTableIndexTime": 7.9999999999999996e-06,
    "symbolTableLoadedFromCache": false,
    "symbolTableParseTime": 7.8999999999999996e-05,
    "symbolTableSavedToCache": false,
    "triple": "arm64-apple-macosx12.0.0",
    "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3"
  },

And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats:

"modules": [
  {
    "debugInfoByteSize": 0,
    "debugInfoIndexLoadedFromCache": false,
    "debugInfoIndexSavedToCache": false,
    "debugInfoIndexTime": 0,
    "debugInfoParseTime": 0,
    "identifier": 4603526968,
    "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out",
    "symbolFileModuleIdentifiers": [
      4604429832
    ],
    "symbolTableIndexTime": 7.9999999999999996e-06,
    "symbolTableLoadedFromCache": false,
    "symbolTableParseTime": 0.000112,
    "symbolTableSavedToCache": false,
    "triple": "arm64-apple-macosx12.0.0",
    "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569"
  },

And the .o file for 4604429832 looks like:

{
  "debugInfoByteSize": 1028,
  "debugInfoIndexLoadedFromCache": false,
  "debugInfoIndexSavedToCache": false,
  "debugInfoIndexTime": 0,
  "debugInfoParseTime": 6.0999999999999999e-05,
  "identifier": 4604429832,
  "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o",
  "symbolTableIndexTime": 0,
  "symbolTableLoadedFromCache": false,
  "symbolTableParseTime": 0,
  "symbolTableSavedToCache": false,
  "triple": "arm64-apple-macosx"
}

Diff Detail

Event Timeline

clayborg created this revision.Feb 9 2022, 7:12 PM
clayborg requested review of this revision.Feb 9 2022, 7:12 PM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 9 2022, 7:12 PM
labath accepted this revision.Feb 10 2022, 2:02 AM

Seems fine. I might consider changing the type of the __dof_cache section (whatever that is) instead of making an exception for eSectionTypeDebug. It don't see anything depending on it, and the exception looks weird.

This revision is now accepted and ready to land.Feb 10 2022, 2:02 AM

Seems fine. I might consider changing the type of the __dof_cache section (whatever that is) instead of making an exception for eSectionTypeDebug. It don't see anything depending on it, and the exception looks weird.

I pinged some folks at Apple to ask what this section is and if we can better set a section type for it. As soon as I hear back, I will make a patch for this and fix it. For now since no one uses .debug sections in COFF files when it comes to statistics, I want the "statistics dump" command to produce accurate output. But I will submit a patch as soon as I hear back about __dof_cache