This is an archive of the discontinued LLVM Phabricator instance.

Add Debug Info Size to Symbol Status
ClosedPublic

Authored by aelitashen on Jul 13 2020, 4:44 PM.

Details

Summary

If a module has debug info, the size of debug symbol will be displayed after the Symbols Loaded Message for each module in the VScode modules view.{F12335461}

Diff Detail

Event Timeline

aelitashen created this revision.Jul 13 2020, 4:44 PM
Herald added a project: Restricted Project. · View Herald TranscriptJul 13 2020, 4:44 PM
wallace requested changes to this revision.Jul 13 2020, 4:51 PM

Please include a test case

lldb/tools/lldb-vscode/JSONUtils.cpp
393–404

Move this logic to another function, so that this function is simpler

398

debug_info is int, thus, if you do debug_info/10, then the result with be an int rounded down. If you cast it to double, you'll have a double without decimals. The correct way to do is to do double(debug_info) / 10.

415

This is a method, therefore it should start with a verb

This revision now requires changes to proceed.Jul 13 2020, 4:51 PM
aelitashen edited the summary of this revision. (Show Details)Jul 13 2020, 4:53 PM
clayborg requested changes to this revision.Jul 13 2020, 11:23 PM

See inlined comments. You will also need to fix the test for this since it will now fail as the "symbolStatus" now contains the size.

lldb/tools/lldb-vscode/JSONUtils.cpp
384–391

Move these lines to a static function above this function:

static uint64_t GetDebugInfoSize(lldb::SBModule module) {
  uint64_t debug_info = 0;
  size_t num_sections = module.GetNumSections();
  for (int i = 0; i < (int)num_sections; i++) {
    lldb::SBSection section = module.GetSectionAtIndex(i);
    debug_info += DebugInfoInSection(section);
  }
  return debug_info;
}
415

Rename this as Walter commented and make this function static. You will need to move it above the function above otherwise the compiler won't see it:

static uint64_t GetDebugInfoSize(lldb::SBSection section) {
416

You need to check if the section is a debug info section by checking if it starts with ".debug_" or "__debug":

uint64_t debug_info_size = 0;
llvm::StringRef section_name(section.GetName());
if ((section_name.startswith(".debug") || section_name.startswith("__debug"))
  debug_info_size += section.GetFileByteSize();
418

Use "size_t" as the type for i and remove (int) cast.

419–420

This will add the section size of all sections that are contained in a section. We don't want this. We want to recursively call this function and have it compare the section name as mentioned above in the inline comments:

debug_info_size += GetDebugInfoSize(section.GetSubSectionAtIndex(i));
lldb/tools/lldb-vscode/JSONUtils.h
446

This isn't a JSON related function. Just make this a static function in the file that uses it and nothing in a header file.

clayborg added inline comments.Jul 13 2020, 11:23 PM
lldb/tools/lldb-vscode/JSONUtils.cpp
393

increase size to 32 just in case we get really big debug info later..

395

Use the PRIu64 macro here to make sure this works on all platforms and the units are wrong here, should just be "B" instead of "KB". Also use snprintf for safety:

snprintf(debug_info_size, sizeof(debug_info_size), " (%"  PRIu64 "B)", debug_info);

PRIu64 is a macro that will expand to a string that will always match a uint64_t. The three strings (" (%", PRIu64, and "B)") will get combined by the compiler into a single format string.

397

no need to multiply the debug_info by 10 here. You will want to make a local double so we don't convert back to an integer:

double kb = (double)debug_info/1024.0;
398

Just use the local double above "kb" and use snprintf:

snprintf(debug_info_size, sizeof(debug_info_size), " (%.1fKB)", kb);
400

Don't multiply by 10 and use a local double:

double mb = (double)debug_info/(1024.0*1024.0);
401

Use local double and use snprintf:

snprintf(debug_info_size, sizeof(debug_info_size), " (%.1fMB)", mb);
403
double gb = (double)debug_info/(1024.0*1024.0*1024.0);
404

Use local and snprintf:

snprintf(debug_info_size, sizeof(debug_info_size), " (%.1fGB)", gb);
422
return debug_info_size;
aprantl added inline comments.Jul 15 2020, 1:41 PM
lldb/tools/lldb-vscode/JSONUtils.cpp
395

Could we avoid snprintf altogether by switching to http://llvm.org/doxygen/classllvm_1_1raw__string__ostream.html ?

Add Test for Debug Info Size

clayborg requested changes to this revision.Jul 16 2020, 1:40 PM
clayborg added inline comments.
lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
44–46

revert renaming and there will be no diffs here

51–53

revert and there will be no diffs here

57–60

We need a comment here if this is for Mac only stating something like:

# On darwin, if we add the unstripped executable as the symbol file, it will end up using the executable and the .o files as the debug info. We won't see any debug information size for this case since all of the debug info sections are in the .o files.
lldb/tools/lldb-vscode/JSONUtils.cpp
333

Need to also check for ".apple" and "__apple" for the Apple DWARF accelerator tables.

336

remove mylog stuff

338–339

remove

340–342

remove mylog stuff

353–354

Remove local "section":

debug_info_size += GetDebugInfoSizeInSection(module.GetSectionAtIndex(i));
This revision now requires changes to proceed.Jul 16 2020, 1:40 PM

The tests added are only for macOS, Walter will help me write tests on Linux. This update optimize the code and apply git clang format.

Herald added a project: Restricted Project. · View Herald TranscriptJul 16 2020, 4:43 PM
Herald added a subscriber: cfe-commits. · View Herald Transcript
wallace added inline comments.Jul 16 2020, 4:47 PM
lldb/tools/lldb-vscode/JSONUtils.cpp
333–335

you can merge these two ifs

clayborg requested changes to this revision.Jul 16 2020, 5:56 PM

So you should revert any clang format changes that are not in modified lines of source, mostly revert a lot of lines in lldb-vscode.cpp.

lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
19

Name should be more descriptive. Maybe "setup_test_common" is a better name

19

So all of these tests can re-use this function if we switch it up a bit. Here is what I was thinking:

def run_test(self, symbol_basename, expect_debug_info_size):
    program_basename = "a.out.stripped"
    program = self.getBuildArtifact(program_basename)
    self.build_and_launch(program)
    functions = ['foo']
    breakpoint_ids = self.set_function_breakpoints(functions)
    self.assertEquals(len(breakpoint_ids), len(functions), 'expect one breakpoint')
    self.continue_to_breakpoints(breakpoint_ids)
    active_modules = self.vscode.get_active_modules()
    program_module = active_modules[program_basename]
    self.assertIn(program_basename, active_modules, '%s module is in active modules' % (program_basename))
    self.assertIn('name', program_module, 'make sure name is in module')
    self.assertEqual(program_basename, program_module['name'])
    self.assertIn('path', program_module, 'make sure path is in module')
    self.assertEqual(program, program_module['path'])
    self.assertTrue('symbolFilePath' not in program_module, 'Make sure a.out.stripped has no debug info')
    self.assertEqual('Symbols not found.', program_module['symbolStatus'])
    symbols_path = self.getBuildArtifact(symbol_basename)
    self.vscode.request_evaluate('`%s' % ('target symbols add -s "%s" "%s"' % (program, symbols_path)))

    def checkSymbolsLoaded():
        active_modules = self.vscode.get_active_modules()
        program_module = active_modules[program_basename]
        return 'Symbols loaded.' == program_module['symbolStatus']

    def checkSymbolsLoadedWithSize():
        active_modules = self.vscode.get_active_modules()
        program_module = active_modules[program_basename]
        symbolsStatus = program_module['symbolStatus']
        symbol_regex = re.compile(r"Symbols loaded. \([0-9]+(\.[0-9]*)?[KMG]?B\)")
        return symbol_regex.match(program_module['symbolStatus'])
            
    if expect_debug_info_size:
        self.waitUntil(checkSymbolsLoadedWithSize)
    else:
        self.waitUntil(checkSymbolsLoaded)

Then your tests would be:

@skipIfWindows
@skipIfRemote    
def test_module_event(self):
    # Mac or linux.

    # On mac, if we load a.out as our symbol file, we will use DWARF with .o files and we will
    # have debug symbols, but we won't see any debug info size because all of the DWARF
    # sections are in .o files.

    # On other platforms, we expect a.out to have debug info, so we will expect a size.
    expect_debug_info_size = platform.system() != 'Darwin'
    return run_test("a.out", expect_debug_info_size)

@skipIfWindows
@skipUnlessDarwin
@skipIfRemote    
def test_module_event_dsym(self):
    # Darwin only test with dSYM file.

    # On mac, if we load a.out.dSYM as our symbol file, we will have debug symbols and we
    # will have DWARF sections added to the module, so we will expect a size.
    return run_test("a.out.dSYM", True)

This should cover both mac and non windows correctly.

20

add a space after program:

program = self.getBuildArtifact(program_basename)
32

Remove @skipUnlessDarwin here. This should work on linux.

59

wrap comment to 80 chars

lldb/tools/lldb-vscode/JSONUtils.cpp
335

yes, merge the if statements

This revision now requires changes to proceed.Jul 16 2020, 5:56 PM

@clayborg, the tests will fail on linux because the Makefile is expecting a .dsym file. I think it's okay if she does it just for Darwin and then I update the test to cover also linux, as I have my linux already well set up.

Create help function in tests for code re-use. Take care of both macOS and Linux system.

Merge Codes and Improve Readability

Merge if statements in GetDebugInfoSizeInSection()

wallace requested changes to this revision.Jul 17 2020, 3:30 PM

The logic looks very good, just some final comments and the code will be high quality

lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
85

With you current Makefile, this test will fail on Linux, as the Makefile will expect a .dsym file to be created. Simply put @ skipUnlessDarwin back here, and add this comment

#TODO: Update the Makefile so that this test runs on Linux

Once this is committed, I'll work on make this test pass on Linux

86–92

The common way to write function comments is with ''', like this

'''
  Mac or linux.

  On mac, if we load a.out as our symbol file, we will use DWARF with .o files and we will
  have debug symbols, but we won't see any debug info size because all of the DWARF
  sections are in .o files.

  On other platforms, we expect a.out to have debug info, so we will expect a size.
'''

That way you don't need to type that many #

100–103

same here about the comment

lldb/tools/lldb-vscode/JSONUtils.cpp
333

apply the format change it suggests, i.e start the second line with ||

353

ConvertDebugInfoSizeToString is a better name

354–368

a more modern way to implement this is

#include <sstream>
#include <iomanip>
 ...
std::ostringstream oss;
oss << "(";
oss << std::fixed << std::setprecision(1);

if (debug_info < 1024) {
  oss << debug_info << "B";
} else if (debug_info < 1024 * 1024) {
  double kb = double(debug_info) / 1024.0;
  oss << kb << "KB";
} else if (debug_info < 1024 * 1024 * 1024) {
  double mb = double(debug_info) / (1024.0 * 1024.0);
  oss << mb << "MB";
} else {
  double gb = double(debug_info) / (1024.0 * 1024.0 * 1024.0);
  oss << gb << "GB";;
}
oss << ")";
return oss.str();

It's actually safer, as you don't need to specify the array size of your debug_info_size_buffer

386–387
symbol_str += ConvertDebugInfoSizeToString(debug_info);

is more concise

This revision now requires changes to proceed.Jul 17 2020, 3:30 PM

Add TODO comment for Linux, Use ostringstream for Debug Info Size message.

wallace accepted this revision.Jul 17 2020, 4:54 PM
wallace added inline comments.
lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
84

remove this whitespace

Remove white space for test_module_event

wallace accepted this revision.Jul 20 2020, 10:49 AM
clayborg requested changes to this revision.Jul 20 2020, 2:40 PM

Just a space before the '(' of the debug info size.

lldb/tools/lldb-vscode/JSONUtils.cpp
355

Need as space before the '(' character as we append this string to the "Symbols loaded." string.

This revision now requires changes to proceed.Jul 20 2020, 2:40 PM

Add a white space in Symbols loaded info.

clayborg accepted this revision.Jul 20 2020, 2:58 PM

Looks good!

This revision is now accepted and ready to land.Jul 20 2020, 2:58 PM

Fix the accidentally removed test contents

clayborg requested changes to this revision.Jul 22 2020, 3:42 PM

Looks like there is an indentation issue in the test. See inline comments.

lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
74–80

Unindent everything after the self.waitUntil()? Otherwise we are not testing the program, symbolFilePath and addressRange on symbols with size.

This revision now requires changes to proceed.Jul 22 2020, 3:42 PM
aelitashen marked an inline comment as done.Jul 23 2020, 12:38 PM
aelitashen added inline comments.
lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
74–80

When unindenting these codes, the dsym test for darwin fails as the symbol paths don't match. One is using dysm path, one is using a.out.path.

clayborg added inline comments.Jul 23 2020, 12:43 PM
lldb/test/API/tools/lldb-vscode/module/TestVSCode_module.py
74–80

Then we need to fix this function so that it does work. One thing to note in the dSYM case: dSYM files are bundles which means it is a directory that contains a file within it. So you might specify "/tmp/a.out.dSYM" as the path, but end up with "/tmp/a.out.dSYM/Context/Resources/DWARF/a.out" as the symbol path. So you could switch the symbol file path to use assertIn:

self.assertIn(symbols_path, program_module['symbolFilePath'])

Fix run_test to cover every system case and attributes

clayborg accepted this revision.Jul 23 2020, 2:39 PM
This revision is now accepted and ready to land.Jul 23 2020, 2:39 PM
This revision was automatically updated to reflect the committed changes.