This is an archive of the discontinued LLVM Phabricator instance.

[analyzer] Tests for scan-build?
AbandonedPublic

Authored by NoQ on Dec 13 2018, 3:53 PM.

Details

Summary

Add a trivial testing infrastructure that could allow writing regression tests for the scan-build tool in case we ever try to improve it.

It's a "before i forget" kind of thing - i played with it a little bit but didn't go far, but then @aaron.ballman pointed out that he could have made some tests for D55665 if the infrastructure was in place.

An obvious next step is to figure out how to test scan-build output, which is annoying because it auto-creates a non-deterministic sub-directory and then produces random file names. I guess we could do wildcards, but we could also make a deterministic output mode; we already have a mode for deterministic file names, so only the date in the directory name is a problem.

Another nice thing to try here is to make lit automatically run the same tests for scan-build-py as well. I expect the %scan_build substitution to allow such tricks, similarly to how we run all Analyzer tests under the Z3 solver if it's enabled.

Diff Detail

Event Timeline

NoQ created this revision.Dec 13 2018, 3:53 PM
Szelethus accepted this revision.Dec 15 2018, 2:38 AM

I don't really use scan-build, so I can't barrage you with suggestions, but the patch looks great.

This revision is now accepted and ready to land.Dec 15 2018, 2:38 AM

Could we make use of scan-build -o to specify the exact output location, in conjunction with stable-report-filename=true to remove the non-determinism? Also, I like the idea of maybe someday making scan-build-py test the same file. However, it seems like that project is basically unmaintained so it may effort to reach parity before that can be enabled.

Thank you for working on the testing infrastructure for this! I think it's super helpful because it gives us a path forward to eventually rewrite scan-build to be more appropriate for all the platforms Clang runs on (like Windows, where the reliance on Perl is a serious hurdle).

NoQ planned changes to this revision.Mar 14 2019, 5:13 PM

This doesn't seem to work anymore.
Also why did i even use $PATH?

Herald added a project: Restricted Project. · View Herald TranscriptMar 14 2019, 5:13 PM
Herald added a subscriber: Charusso. · View Herald Transcript