This is an archive of the discontinued LLVM Phabricator instance.

[analyzer] support a mode to only show relevant lines in HTML diagnostics
ClosedPublic

Authored by george.karpenkov on Dec 18 2017, 7:04 PM.

Details

Summary

HTML diagnostics can be an overwhelming blob of pages of code.
This patch adds a checkbox which filters this list down to only the lines *relevant* to the counterexample by
e.g. skipping branches which analyzer has assumed to be infeasible at a time.

The resulting amount of output is much smaller, and often fits on one screen, and also provides a much more readable diagnostics.

Diff Detail

Repository
rC Clang

Event Timeline

Applying clang-format.

Bugfixes, shortcut, always showing function prototype.

Proper macro handling.

Added tests. It is bad to match against HTML output directly, but we can match against provided JSON.

george.karpenkov edited the summary of this revision. (Show Details)
NoQ added a comment.Jan 3 2018, 11:12 AM

I'm liking how it works!

include/clang/StaticAnalyzer/Core/BugReporter/PathDiagnostic.h
112–114

It might be a good moment to figure out which of these are actually used. Do we use Extensive and AlternateExtensive at all anywhere?

lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1910–1912 ↗(On Diff #127772)

Note that neither BlockDecl nor ObjCMethodDecl is a FunctionDecl.

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
51

The order in which lines are listed would not be deterministic (right?), but i don't particularly care (@xazax.hun, is it important for your stuff?).

george.karpenkov added inline comments.
include/clang/StaticAnalyzer/Core/BugReporter/PathDiagnostic.h
112–114

PlistDiagnosticuses Extensive (strange, since I've expected the output to be conceptually the same as what we get in HTML), and enabling path-diagnostics-alternate option gets AlternateExtensive

lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1910–1912 ↗(On Diff #127772)

I don't think we should care about blocks at this point.
Added ObjcMethodDecl support.

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
51

There is a bunch of non-deterministic outputs in HTML report already, but since we are trying to reduce non-determinism I think we might as well just use map/set here.

NoQ accepted this revision.Jan 5 2018, 6:32 PM

I think this should land. Hope nobody minds some interactivity.

lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1905–1908 ↗(On Diff #128581)

Even though this works, i guess you could simply see if the current program point is CallEnter or CallExit or something like that, and extract all the info you wanted, instead of doing this every time you don't have a statement.

1930 ↗(On Diff #128581)

Maybe getOpaqueValue() would express what you intend (i.e. lack of collisions) better, even though i'd be shocked if it ever mattered.

This revision is now accepted and ready to land.Jan 5 2018, 6:32 PM

At a high level it seems pretty weird that the relevant lines for a diagnostic is represented a path diagnostic piece. This information is not a piece of the path; rather it is metadata about the entire path. It seems to me like this would be better represented as part of the path diagnostic itself and not pieces.

test/Analysis/relevant_lines/header.h
1 ↗(On Diff #127772)

Can you put in "relevant_lines" directory into a sub directory of test/Analysis called "html_diagnostics" or something like that? This will make it clear that people shouldn't look into "relevant_lines" to try to understand the analysis parts for the analyzer.

test/Analysis/relevant_lines/macros_same_file.c
13 ↗(On Diff #127772)

It is not clear to me why you can't use FileCheck for this. I think that would be greatly preferable to adding and maintaining a new test helper executable.

@dcoughlin your high-level comment makes sense, I wanted a least intrusive change.
What about just adding a metadata field to PathDiagnostic?

@dcoughlin your high-level comment makes sense, I wanted a least intrusive change.
What about just adding a metadata field to PathDiagnostic?

That seems reasonable to me (although note that we already have an *unstructured* text collection (sigh) of metadata (see meta_begin()). If you took this approach you could add a "relevantLines" (or whatever) field to PathDiagnostic and populate that field when traversing the path. Then it would be up to the path diagnostic consumer to turn relevant lines into the a visualization (for the HTML consumer it would be your JavaScript; for the plist consumer we could just emit the dictionary in plist form and let clients visualize it, etc.).

george.karpenkov marked 2 inline comments as done.Jan 10 2018, 3:34 PM
george.karpenkov added inline comments.
lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1930 ↗(On Diff #128581)

I'm not sure what do you mean here.

NoQ added inline comments.Jan 10 2018, 3:38 PM
lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1930 ↗(On Diff #128581)

Like, FID.getHashValue() kinda sounds to me like "compute a small hash of the large FID object, which may potentially collide with other FID objects", while FID.getOpaqueValue() sounds like "give me the whole sequence of bytes in FID that uniquely identifies it".

Changed: now ExecutedLines field belongs to PathDiagnostic and is always calculated. As a result, the change is much smaller.

lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1930 ↗(On Diff #128581)

Makes sense, but getOpaqueValue is private.
Could we expose it?

NoQ added inline comments.Jan 10 2018, 4:40 PM
lib/StaticAnalyzer/Core/BugReporterVisitors.cpp
1930 ↗(On Diff #128581)

Whoooops. Never mind then.

@dcoughlin @NoQ further thoughts? Good to go?

NoQ accepted this revision.Jan 12 2018, 4:15 PM

Looks great.

lib/StaticAnalyzer/Core/BugReporter.cpp
3520–3528

Signature->getBody()?

Thanks! This is quite nice.

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
523

Please move the shortcut documentation to the keyboard navigation help hover that you added in r321320 before committing. Most users won't use the shortcut, so it would be good not to clutter up the main user interface with documentation for it.

It would also be good to not reuse the macro expansion CSS class and instead create new CSS classes for the help hover. This way changes to the macro expansion visualization won't accidentally affect the help visualization and vice versa.

545

Should this be ": 1" or some other truthy value? Doesn't using 0 mean we'll never see the relevant lines when the box is checked?

NoQ added inline comments.Jan 16 2018, 1:57 PM
lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
545

Hmm, indeed, it might be possible to turn this into a list, and also later iterate over that list rather than over all line elements. Performance can be easily evaluated on any html report for a preprocessed sqlite3.c file (:

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
523

OK would that change require a separate review?

545

:0 just emulates a set from a map. E.g. Java does it with :null.
Iterating over list might be faster, but then we would still need to look up the DOM element from the line number.

dcoughlin added inline comments.Jan 16 2018, 6:03 PM
lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
523

No.

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
545

actually iterating over a list would not be any better as we would still run in time proportional to the overall number of lines. Committing as it is, with the shortcut documentation (re)moved.

This revision was automatically updated to reflect the committed changes.