This is an archive of the discontinued LLVM Phabricator instance.

[DWARF] Rework debug line parsing to use llvm::Error
AbandonedPublic

Authored by jhenderson on Mar 12 2018, 7:14 AM.

Details

Summary

The .debug_line parser previously reported errors by printing to stderr and return false. This is not particularly helpful for clients of the library code, as it prevents them from handling the errors in a manner based on the calling context. This change switches to using llvm::Error to indicate what problems were detected during parsing, and has updated clients to handle the errors in a location-specific manner. In general, this means that they continue to do the same thing to external users. Below, I have outlined what the known behaviour changes are, relating to this change.

There are three levels of "errors" in the new error mechanism, to broadly distinguish between different fail states of the parser, since not every failure will prevent parsing of the unit, or of subsequent unit. Fatal errors represent errors that prevent reading the unit length field. If this happens, it will be impossible to know where the next unit starts, so this will prevent further parsing of both the current unit and any subsequent ones. Major errors represent errors that prevent reading the remainder of the table, e.g. because
the version is unsupported. Minor errors represent problems with parsing that do not prevent attempting to continue reading the table. The only example of this currently is when the last sequence of a unit is unterminated. However, I think it would be good to change the handling of unrecognised opcodes to report as minor errors as well, rather than just printing to the stream if --verbose is used (this would be a subsequent change however).

I have substantially extended the DwarfGenerator to be able to handle custom-crafted .debug_line sections, allowing for comprehensive unit-testing of the parser code. For now, I am just adding unit tests to cover the basic error reporting, and positive cases, and do not currently intend to test every part of the parser, although the framework should be sufficient to do so at a later point. Note that the current diff is still a work-in-progress in the unit tests - I am happy with most of what is there, but I intend to add further tests for basic DWARF 5 positive cases, and for each of the different possible errors that can be reported. I will update the diff once these are complete, but I wanted to put the current state up so that people can start commenting on it.

Known behaviour changes:

  • The dump function in DWARFContext now does not attempt to read subsequent tables when searching for a specific offset, if there was a fatal error.
  • The dump function now uses the severity of the error to determine if subsequent units should be read or not. If a major error is detected, the table is skipped, but subsequent ones are read, making the assumption that the unit length field is valid.
  • getOrParseLineTable now returns a useful Error if an invalid offset is encountered, rather than simply a nullptr.
  • getOrParseLineTable now returns both the Error returned from LineTable::parse and the line table, if only Minor-level errors are encountered.
  • The parse functions no longer use fprintf(stderr,...) to report errors, meaning that LLD will now correctly print errors, rather than them sometimes not being flushed, or being interleaved with other errors (see also the forthcoming LLD review).
  • The existing parse error messages have been updated to not specifically include "warning" in their message, allowing consumers to determine what severity the problem is.
  • If the line table version field appears to have a value less than 2, an informative error is returned, instead of just false.
  • If the line table unit length field uses a reserved value, an informative error is returned, instead of just false.

As a helper for the generator code, I have re-added EmitInt64 to the AsmPrinter code. This previously existed, but was removed way back in r100296, presumably because it was dead at the time. A quick review of LLVM code suggests that there are several places that could do with this function, instead of using EmitIntValue(..., 8).

This change also requires a change to LLD. I will post a separate review for this shortly, since I don't know how to reliably create a review for both repositories simultaneously.

I'm conscious that this is quite a large change, so if anybody has any suggestions on how to usefully break it up, please let me know.

Diff Detail

Repository
rL LLVM

Event Timeline

jhenderson created this revision.Mar 12 2018, 7:14 AM

I am quite new to this part of llvm, so I'll just sit back and learn. I'll just say that a similar error classification would be very useful for the debug_names parser (which I am working on now, and I have noticed the inability to signal that just one of the name indexes is corrupt) and probably a lot of other debug info parsers, so I welcome this effort (though I'm not entirely sure that your usage of llvm::Error is completely idiomatic here).

include/llvm/DebugInfo/DWARF/DWARFDebugLine.h
68

Drop the &&. This makes it clear that the function will clear the error (whereas previously it could just ignore them and leave the caller with an uncleared error).

lib/DebugInfo/DWARF/DWARFDebugLine.cpp
42–43

Why should we ever have an error with severity "none"?

(though I'm not entirely sure that your usage of llvm::Error is completely idiomatic here).

Thanks for the comments. I am fairly new to llvm::Error myself, so I completely agree that this is probably a little unusual. I did consider the alternative which is to have a separate ErrorInfo class for each of the different severities, but that didn't fit particularly well in some situations, e.g. when treating Major amd Fatal errors the same, but distinct from Minor. I'm certainly open to suggestions for alternatives.

lib/DebugInfo/DWARF/DWARFDebugLine.cpp
42–43

You shouldn't (and I'm tempted to put an unreachable assert here). The problem is that we need a default value that indicates no error, so that a caller doesn't also have to check against Error::success().

This change also requires a change to LLD. I will post a separate review for this shortly, since I don't know how to reliably create a review for both repositories simultaneously.

Review posted as D44388.

Hi James,

First off, thanks for working on this! Unfortunately I didn't get around to reviewing this today. As a general note, like Pavel already said, this isn't really idiomatic when it comes to the Error class. The None severity doesn't make sense (you could always return an Optional<Severity> where necessary) but I don't have a good solution in mind when it comes to surfacing the other kind of errors. I'll have a more in-depth look tomorrow to better understand the constraints.

James Henderson via Phabricator via llvm-commits
<llvm-commits@lists.llvm.org> writes:

jhenderson created this revision.
jhenderson added reviewers: probinson, dblaikie, JDevlieghere, aprantl, labath.

The .debug_line parser previously reported errors by printing to stderr and return false. This is not particularly helpful for clients of the library code, as it prevents them from handling the errors in a manner based on the calling context. This change switches to using llvm::Error to indicate what problems were detected during parsing, and has updated clients to handle the errors in a location-specific manner. In general, this means that they continue to do the same thing to external users. Below, I have outlined what the known behaviour changes are, relating to this change.

There are three levels of "errors" in the new error mechanism, to broadly distinguish between different fail states of the parser, since not every failure will prevent parsing of the unit, or of subsequent unit. Fatal errors represent errors that prevent reading the unit length field. If this happens, it will be impossible to know where the next unit starts, so this will prevent further parsing of both the current unit and any subsequent ones. Major errors represent errors that prevent reading the remainder of the table, e.g. because
the version is unsupported. Minor errors represent problems with parsing that do not prevent attempting to continue reading the table. The only example of this currently is when the last sequence of a unit is unterminated. However, I think it would be good to change the handling of unrecognised opcodes to report as minor errors as well, rather than just printing to the stream if --verbose is used (this would be a subsequent change however).

The main advantaged of using Expected is that it is not possible to
access an invalid object.

If possible I think it would be better to reserve Error/Expected for "it
was not possible to produce the requested result" and have some
logging/diagnostics/callback system for things that can be ignored.

For example, getOrParseLineTable could return Expected<const LineTable
*> and take a callback for non fatal issues:

if (Error = Callback(DebugLineDiag::Minor,

                                "last sequence in debug line table is not"
                                " terminated!"))
return Error;

And it is up to the callback whether it wants getOrParseLineTable to fail.

Alternatively it could return Expected<std::pair<const LineTable *, Warnings>>.

Cheers,
Rafael

Dropped unnecessary '&&' usage. Replaced use of "None" Severity with an Optional. Added remaining missing unit tests, fixed a number of issues in the test framework, and refactored some of the common test code to reduce code duplication. Also add paramterised test instantations for different DWARF formats.

I haven't yet attempted in this diff to look at how to make the code more idiomatic. I'll do that next, but I might put it up as an alternative review, so that we can evaluate both approaches simultaneously. One idea I had was rather than to use different Severities associated with each individual DebugLineError instance, to have different ErrorInfo classes for the different severities (e.g. have a DebugLineFatalError, DebugLineMajorError, and DebugLineMinorError). The handling code could still use a Severity enum or similar to report back, or I could just change how the handling code works, to take in a callback, similar to what was suggested. Thoughts?

jhenderson marked 2 inline comments as done.Mar 15 2018, 4:24 AM
jhenderson abandoned this revision.Mar 19 2018, 2:25 AM

I think D44560 is my preferred approach overall. Abandoning.