This option allows printing all sources used by an object file.
Details
Diff Detail
- Repository
- rG LLVM Github Monorepo
Unit Tests
Event Timeline
While this information can be extracted out of the existing llvm-dwarfdump output, it requires additional post-processing. This became so common in our project that we have implemented a custom Go-based tool for that purpose, but that has other downsides such as the lack of DWARF5 support. I think that supporting this option directly in llvm-dwarfdump might be generally useful and it doesn't add a lot of complexity. I'm open to suggestions for how to improve the output.
Sounds alright - few things could be simplified, etc.
llvm/test/tools/llvm-dwarfdump/X86/sources.s | ||
---|---|---|
12–58 | Maybe simplify the functions (to something like simple void/do-nothing functions) to reduce the length of the assembly, no need for types. | |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
462 | Could use std::move(FullPath) here, if you like, but hardly critical. | |
465 | Could use llvm::sort here |
Please add the new option to the Command Guide documentation.
llvm/test/tools/llvm-dwarfdump/X86/sources.s | ||
---|---|---|
9 | You can probably dramatically simplify this code by changing to use yaml2obj. I believe that ELF yaml2obj DWARF support is sufficiently powerful now to achieve this. @Higuoxing may be able to provide more information on this, as he did the work recently there. See also llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml for an example input. | |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
455 | I think we need testing for multiple CUs. The current test only checks a single one. This might go against the yaml2obj usage suggested above though (@Higuoxing, is there support for multiple tables in .debug_line yet?). | |
456 | Not that it likely is going to matter in any practical situation, but this should probably be uint64_t technically - the FileNames are set via LEB128 values (see e.g. DW_LNS_set_file) and thus technically have no upper bound in size from the file format. I won't fight too hard for this if you don't want to though. | |
467 | ||
711 | Not related to this patch, or even something you should do yourself. More idle musing - as llvm-dwarfdump starts gaining moreof these options, it feels like it should be able to do multiple at once (e.g. allow llvm-dwarfdump --show-sources --show-section-sizes). |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 |
Yes, yaml2obj supports emitting multiple line tables. I'm able to help craft these test cases. It looks that LT isn't checked. If a compilation unit doesn't have an associated line table, llvm-dwarfdump --show-sources will crash. const auto *LT = DICtx.getLineTableForUnit(CU.get()); // Can be a null pointer. for (uint32_t I = 1; I <= LT->Prologue.FileNames.size(); ++I) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ... } We can reproduce it using the following test case. $ yaml2obj %s | llvm-dwarfdump --show-sources - --- !ELF FileHeader: Class: ELFCLASS64 Data: ELFDATA2LSB Type: ET_EXEC Machine: EM_X86_64 DWARF: debug_info: - Version: 4 |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | Nice catch! In fact, do we really need to use the CUs at all for this? Could we not just iterate over all line tables? That would allow this to work when there is no .debug_info data too (which the DWARF spec implies is permitted). |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | I don't know how carefully the spec says it is permitted, but certainly I've heard committee members talk about stripping everything but .debug_line (and with v5, .debug_line_str) from an object file. In DWARF v4, technically the primary source file & compilation dir could be omitted from the line table, although in practice I think that never happens. In v5 the primary source file & dir are supposed to be explicit in the line table, so I think ignoring .debug_info ought to be okay in general. |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | @jhenderson is there an API to iterate over all line tables? I searched through LLVM but haven't found anything. |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | I thought there was, but having taking a look, I don't know of an interface that allows you to simply iterate over all line tables without parsing all of them. Certainly you can iterate over all the line tables by parsing them in order by using the SectionParser class of DWARFDebugLine.h. I'm not sure if that's exactly the right way forward here though, since I suspect by this point the DWARFContext may have already done (some of) the parsing (I haven't dug into the code to confirm either way). There's also getOrParseLineTable, which takes an offset, Context and DWARFDataExtractor and gives you back the line table at that offset, which may or may not have already been parsed (it will return the cached version if it has been). You'd need to then use the length field within the line table to identify the next offset to use. Maybe a new function could sit on top of that to give you the ability to iterate over them, and only parse the ones that haven't been already? Alternatively, you could modify the SectionParser class to cache the parsed line tables so that it doesn't matter if you try to reparse them later. |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | I think you are able to iterate over line tables via the following code snippets. DWARFDataExtractor LineData(DICtx.getDWARFObj(), DICtx.getDWARFObj().getLineSection(), DICtx.isLittleEndian(), 0); DWARFDebugLine::SectionParser Parser(LineData, DICtx, DICtx.normal_units()); while (!Parser.done()) { DWARFDebugLine::LineTable LT = Parser.parseNext( RecoverableErrorHandler, UnrecoverableErrorHandler); // Dump file names with paths. ... } | |
456 | I'm not sure if the for-loop should start from 0. The DWARFv5 spec says:
|
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | Thanks, I tried that which made me realize that without CU, we don't have the comp_dir, is that something we care about? |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | I have no idea. Perhaps @jhenderson and @dblaikie can help us? |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | Ah, that's a good point. I think having the compilation directory is useful, but perhaps not a deal breaker. In other words, if it would be clean to do, I'd think the behaviour could be:
I'm very much open to other thoughts though. I feel like this option could be useful without .debug_info being present, but I don't know how much of a common case that actually is. |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
455 | The question is how to iterate over line tables (rather than over CUs and retrieving their line tables)? But you don't want all the parsed data, just the file and line tables? Yeah, looks like the nearest tool available is DWARFDebugLine::SectionParser but, as noted it does seem to do all the parsing up-front. I wouldn't be averse to/would generally encourage refactorings that make APIs like this lazier - parses maybe just a bit of the line table header, then returns - then you can query it for files, directories, and line table entries as desired. Those queries can fail, of course (since parsing hasn't been done up-front), so the query APIs should reflect that possibility. Such refactoring can/should be done separately, with new test cases added - perhaps using unit tests, where, say, a line table with a valid directory list exists, but with invalid data after that - by lazy parsing, it should be possible to query just the directory table without ever reaching the invalid data/getting any errors. Similarly - the ability to minimal-parse one line table and jump to the next one immediately - skipping over some invalidity in the first line table without errors because it's never queried in detail, etc. I made some fixes along these lines to loclist and rnglist parsing in the last week or so for a variety of reasons, for instance. |
Taking over this change from phosek, upon request.
Updated implementation to slurp all available path information from CU
filename, CU include dir, and debug line info.
Pull from file index 0 if valid, otherwise start at 1.
Updated tests to use yaml2obj.
I think your test cases need extending with multiple source names per input, and also what happens if you specify multiple input files (I believe llvm-dwarfdump handles that, but I might be mistaken).
This new option still needs adding to the CommandGuide documentation at llvm/docs/CommandGuide/llvm-dwarfdump.rst.
llvm/test/tools/llvm-dwarfdump/X86/sources.test | ||
---|---|---|
1–3 ↗ | (On Diff #420637) | Up to you, but I have a personal preference for this formatting, as it indicates on each line that there is a continuation involved, starting with a new command. Also, I personally prefer it if tests create objects on disk rather than passing them via stdin. This is because it's easier to directly inspect the binary if there's a problem with the test. Otherwise, you have to (temporarily) modify the test to force it to dump it. Same comments apply elsewhere. |
5 ↗ | (On Diff #420637) | You could simplify this an similar patterns by dropping "CHECK" from the prefix name. This will also match "foobarname.csnfdssfnfjds" which probably isn't the intent. I think as you're testing a new dumping option, you should add the following options to the FileCheck command (also applies below): --match-full-lines --implicit-check-not={{.}} The former effectively wraps the check pattern with {{^}} and {{$}}. If there's any whitespace involved, you can also add --strict-whitespace although I don't think there is here? The second option ensures that only the checked output is emitted and nothing else at all. This ensures there's no output before or after the checked pattern on different lines. |
31 ↗ | (On Diff #420637) | If you are expecting no output, I'd use count 0 instead of FileCheck here as it's stricter. Should this case print a warning though? |
92 ↗ | (On Diff #420637) | This is a good example of something that will pass spuriously without --match-full-lines, since comp/dir/abs/name.c will be successfully matched by it. |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
459–460 | I'm not sure I see testing that exercises both sides of each of the two ternaries in this loop. | |
468–470 | I don't believe that there's a test case for the case where an absolute path hasn't been produced? | |
480 | Don't use auto unless the type is obvious from the immediate context (e.g. it's already specified on the line due to a case): https://llvm.org/docs/CodingStandards.html#use-auto-type-deduction-to-make-code-more-readable | |
483–484 | This last part of the sentence is garbled. | |
485 | Too much auto. | |
492 | The statement isn't correct though: filenames are included in the DWARF line table v4 and earlier... (although not explicitly the compilation directory). | |
505–506 | I'm not convinced that the bit in parentheses is correct, based on the earlier conversation in this review. I don't think it's particularly useful information either. | |
513 | You need a test case to show that this is set by parsing failures in the line table. | |
529 | Too much auto and unnecessary "const &" (StringRef is designed to be trivially copyable). |
Apologies for the long delay; this change slipped my mind for a bit.
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
468–470 | Removed the check here; this was to avoid mixing relative and absolute paths from the CU and line table. But, now that we either get all names from the line table or just the name from the CU, this is no longer an issue. | |
480 | This appears to be one of the cases mentioned as an exception: where the underlying type is abstracted away. I did a quick scan though usages of DICtx.compile_units(); all but two use (const auto &CU). | |
483–484 | Fixed; sorry about that. | |
492 | You're right; this simplifies things quite a bit. I've changed this to only add the name if the linetable is missing; otherwise we can get everything from the linetable. |
llvm/docs/CommandGuide/llvm-dwarfdump.rst | ||
---|---|---|
118–121 ↗ | (On Diff #440412) | Options should be in alphabetical order, so this needs moving above --statistics. |
llvm/test/tools/llvm-dwarfdump/X86/sources.test | ||
7 ↗ | (On Diff #440412) | Should this be CU-NAME-NEXT:? |
46–50 ↗ | (On Diff #440412) |
# RUN: FileCheck -DFILE=%t.comp-dir.err ... # CU-COMP-DIR: warning: [[FILE]]: ... |
87–89 ↗ | (On Diff #440412) | For FileCheck commands, it's fine to pipe stdin directly to the command, rather than going via an intermediate file. |
134 ↗ | (On Diff #440412) | You can leverage yaml2obj's -D option much in the same manner as the FileCheck one above to avoid the need for two (or more) near-identical blocks of YAML, e.g. to provide the file names in posix and windows formats. There may well be other cases within your tests that are similar. |
399–400 ↗ | (On Diff #440412) | Similar to the above comment about piping, pipe the output directly to count here. In general, the rule I follow is: is the output a binary file or similar? If so, stick it in a file. Otherwise, pipe it. |
420 ↗ | (On Diff #440412) | Check the error message. |
1–3 ↗ | (On Diff #420637) | You missed the space indentation I included in my suggested edit. |
Address review comments.
llvm/test/tools/llvm-dwarfdump/X86/sources.test | ||
---|---|---|
1–3 ↗ | (On Diff #420637) | Not sure what you mean here; I'm seeing three space indentation from the colon in RUN: in both the code and your suggested edit. |
llvm/test/tools/llvm-dwarfdump/X86/sources.test | ||
---|---|---|
348 ↗ | (On Diff #440752) | Usually when we're just checking an error or warning message, we use 2>&1 to combine stderr and stdout, rather than checking the two separately. This is especially relevant here, because you do the --implicit-check-not check. |
1–3 ↗ | (On Diff #420637) | Sorry misread the test before (I thought FileCheck was still being passed input by pipe, which it wasn't in that iteration). Latest bit looks good. |
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
459–460 | Did this get addressed (it might have done, but I don't have time to dig into the test coverage)? |
Add dwarfv5 test and use 2>&1 for error checking.
llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp | ||
---|---|---|
459–460 | The second ternary is covered by the %t.no-filenames.o test, but it looks like I missed the first one when I was picking this patch back up. The alternative case of the first ternary can only be triggered in DWARFv5, but it looks like yaml2obj doesn't fully support it for line tables yet. I've added a test through using a compiled object file for this case, with a TODO to use yaml2obj. |
You can probably dramatically simplify this code by changing to use yaml2obj. I believe that ELF yaml2obj DWARF support is sufficiently powerful now to achieve this. @Higuoxing may be able to provide more information on this, as he did the work recently there. See also llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml for an example input.