[scan-build-py] Update scan-build-py to allow outputing as SARIF.

[isthismyaccount]

Summary

clang static analysis reports can be generated in html, plist, or sarif format.
This updates scan-build-py to be able to specify SARIF as the desired output format, as previously it only support plist and html formats.

[aabbaabb]

LGTM

[isthismyaccount]

Update to support running multiple TUs at once and merging the resulting sarif file into a single sarif file with multiple runs.

Support embedded links in results, codeflows, and threadflows. Does not support embedded links in other message properties (eg: location, since these generally do not refer to sarif runs indices)

[isthismyaccount]

Cleanup related to merge_sarif_files to be more readable, update the merge sarif test.

[aabbaabb]

clang/tools/scan-build-py/libscanbuild/report.py

313 ↗

(On Diff #320523)

arr not used?

Change to:

arr = [update_sarif_object(entry, runs_count_offset) for entry in sarif_object[key]]
sarif_object[key] = arr

337 ↗

(On Diff #320523)

I think loop in a reverse order like
for i in range(len(matches)-1, -1, -1):

makes more sense.

347 ↗

(On Diff #320523)

pattern only used above, suggest move it up.

370 ↗

(On Diff #320523)

Here you are assuming all the run index in one sarif file is sequential. (I believe this should be the case)
However, a more general approach is not to pass in a runs_count_offset, but rather, a runs_count
and then runs_count++ after every run in every sarif file. This way, we no longer rely on the original run index.

[isthismyaccount]

clang/tools/scan-build-py/libscanbuild/report.py

370 ↗

(On Diff #320523)

From my understanding, in a single sarif file, runs can point to any index in the set of contained runs. Ie we could have something like:

sarif:

{ 
'runs': [
   { 'message': { 'text': 'this is a link to a future run [link](sarif:/runs/1/result/0)'}},
   { 'message': { 'text': 'this is a link to another future run [link](sarif:/runs/2/result/0'}},
   { 'message': { 'text': 'this is a link to the first run [link](sarif:/runs/0/result/0'}}
]
}

In this case, when merging this set of runs into another set of runs, embedded links would be updated to be relative to the "base" run index, such that and index of 0 becomes base_run_index, 1 becomes base_run_index+1, etc. I'm unsure of whether or not I can achieve this by iteratively increasing runs_count, since run indices are initially 0-indexed.

From
https://docs.oasis-open.org/sarif/sarif/v2.0/csprd02/sarif-v2.0-csprd02.html#_Toc10127658

Because the "sarif" URI scheme uses JSON pointer [RFC6901], which locates array elements by their array index, these URIs are potentially fragile if the SARIF log file is transformed by a post-processor.

EXAMPLE 2: If a post-processor concatenates two runs into a single log file, the links within the run at index 1 will be incorrect, and will need to be updated from "sarif:/runs/0/…" to "sarif:/runs/1/…".

[isthismyaccount]

Update based on aabbaabb's comments.

[aabbaabb]

clang/tools/scan-build-py/libscanbuild/report.py

370 ↗

(On Diff #320523)

I see.

[aabbaabb]

lgtm

clang/tools/scan-build-py/libscanbuild/analyze.py

 
 
 @command_entry_point
 def scan_build():
     """ Entry point for scan-build command. """
 
     args = parse_args_for_scan_build()
     # will re-assign the report directory as new output
-    with report_directory(args.output, args.keep_empty) as args.output:
+    with report_directory(
+            args.output, args.keep_empty, args.output_format) as args.output:
         # Run against a build command. there are cases, when analyzer run
         # is not required. But we need to set up everything for the
         # wrappers, because 'configure' needs to capture the CC/CXX values
         # for the Makefile.
         if args.intercept_first:
             # Run build command with intercept module.
             exit_code = capture(args)
             # Run the analyzer against the captured commands.

 
 
 @command_entry_point
 def analyze_build():
     """ Entry point for analyze-build command. """
 
     args = parse_args_for_analyze_build()
     # will re-assign the report directory as new output
-    with report_directory(args.output, args.keep_empty) as args.output:
+    with report_directory(args.output, args.keep_empty, args.output_format) as args.output:
         # Run the analyzer against a compilation db.
         govern_analyzer_runs(args)
         # Cover report generation and bug counting.
         number_of_bugs = document(args)
         # Set exit status as it was requested.
         return number_of_bugs if args.status_bugs else 0
 
 
for source in compilation.files:
         current = run(parameters)
         # display error message from the static analyzer
         if current is not None:
             for line in current['error_output']:
                 logging.info(line.rstrip())
 
 
 @contextlib.contextmanager
-def report_directory(hint, keep):
+def report_directory(hint, keep, output_format):
     """ Responsible for the report directory.
 
     hint -- could specify the parent directory of the output directory.
     keep -- a boolean value to keep or delete the empty report directory. """
 
     stamp_format = 'scan-build-%Y-%m-%d-%H-%M-%S-%f-'
     stamp = datetime.datetime.now().strftime(stamp_format)
     parent_dir = os.path.abspath(hint)
     if not os.path.exists(parent_dir):
         os.makedirs(parent_dir)
     name = tempfile.mkdtemp(prefix=stamp, dir=parent_dir)
 
     logging.info('Report directory created: %s', name)
 
     try:
         yield name
     finally:
         if os.listdir(name):
+            if output_format != 'sarif':
+                # 'scan-view' currently does not support sarif format.
                 msg = "Run 'scan-view %s' to examine bug reports."
+            else:
+                msg = "View result at %s/result.sarif."
             keep = True
         else:
             if keep:
                 msg = "Report directory '%s' contains no report, but kept."
             else:
                 msg = "Removing directory '%s' because it contains no report."
         logging.warning(msg, name)
 

 
 @require(['command',  # entry from compilation database
           'directory',  # entry from compilation database
           'file',  # entry from compilation database
           'clang',  # clang executable name (and path)
           'direct_args',  # arguments from command line
           'force_debug',  # kill non debug macros
           'output_dir',  # where generated report files shall go
-          'output_format',  # it's 'plist', 'html', both or plist-multi-file
+          'output_format',  # it's 'plist', 'html', 'plist-html', 'plist-multi-file', or 'sarif'
           'output_failures',  # generate crash reports or not
           'ctu'])  # ctu control options
 def run(opts):
     """ Entry point to run (or not) static analyzer against a single entry
     of the compilation database.
 
     This complex task is decomposed into smaller methods which are calling
     each other in chain. If the analysis is not possible the given method
def target():
                 'plist',
                 'plist-html',
                 'plist-multi-file'}:
             (handle, name) = tempfile.mkstemp(prefix='report-',
                                               suffix='.plist',
                                               dir=opts['output_dir'])
             os.close(handle)
             return name
+        elif opts['output_format'] == 'sarif':
+            return os.path.join(opts['output_dir'], 'result.sarif')
         return opts['output_dir']
 
     try:
         cwd = opts['directory']
         cmd = get_arguments([opts['clang'], '--analyze'] +
                             opts['direct_args'] + opts['flags'] +
                             [opts['file'], '-o', target()],
                             cwd)

clang/tools/scan-build-py/libscanbuild/arguments.py

format_group.add_argument(
         '--plist-multi-file',
         '-plist-multi-file',
         dest='output_format',
         const='plist-multi-file',
         default='html',
         action='store_const',
         help="""Cause the results as a set of .plist files with extra
         information on related files.""")
+    format_group.add_argument(
+        '--sarif',
+        '-sarif',
+        dest='output_format',
+        const='sarif',
+        default='html',
+        action='store_const',
+        help="""Cause the results as a result.sarif file.""")
 
     advanced = parser.add_argument_group('advanced options')
     advanced.add_argument(
         '--use-analyzer',
         metavar='<path>',
         dest='clang',
         default='clang',
         help="""'%(prog)s' uses the 'clang' executable relative to itself for

clang/tools/scan-build-py/tests/unit/test_analyze.py

class Spy(object):
     def call(self, params):
         self.arg = params
         return self.success
 
 
 class RunAnalyzerTest(unittest.TestCase):
 
     @staticmethod
-    def run_analyzer(content, failures_report):
+    def run_analyzer(content, failures_report, output_format='plist'):
         with libear.TemporaryDirectory() as tmpdir:
             filename = os.path.join(tmpdir, 'test.cpp')
             with open(filename, 'w') as handle:
                 handle.write(content)
 
             opts = {
                 'clang': 'clang',
                 'directory': os.getcwd(),
                 'flags': [],
                 'direct_args': [],
                 'file': filename,
                 'output_dir': tmpdir,
-                'output_format': 'plist',
+                'output_format': output_format,
                 'output_failures': failures_report
             }
             spy = Spy()
             result = sut.run_analyzer(opts, spy.call)
-            return (result, spy.arg)
+            output_files = []
+            for entry in os.listdir(tmpdir):
+                output_files.append(entry)
+            return (result, spy.arg, output_files)
 
     def test_run_analyzer(self):
         content = "int div(int n, int d) { return n / d; }"
-        (result, fwds) = RunAnalyzerTest.run_analyzer(content, False)
+        (result, fwds, _) = RunAnalyzerTest.run_analyzer(content, False)
         self.assertEqual(None, fwds)
         self.assertEqual(0, result['exit_code'])
 
     def test_run_analyzer_crash(self):
         content = "int div(int n, int d) { return n / d }"
-        (result, fwds) = RunAnalyzerTest.run_analyzer(content, False)
+        (result, fwds, _) = RunAnalyzerTest.run_analyzer(content, False)
         self.assertEqual(None, fwds)
         self.assertEqual(1, result['exit_code'])
 
     def test_run_analyzer_crash_and_forwarded(self):
         content = "int div(int n, int d) { return n / d }"
-        (_, fwds) = RunAnalyzerTest.run_analyzer(content, True)
+        (_, fwds, _) = RunAnalyzerTest.run_analyzer(content, True)
         self.assertEqual(1, fwds['exit_code'])
         self.assertTrue(len(fwds['error_output']) > 0)
 
+    def test_run_analyzer_with_sarif(self):
+        content = "int div(int n, int d) { return n / d; }"
+        (result, fwds, output_files) = RunAnalyzerTest.run_analyzer(content, False, output_format='sarif')
+        self.assertEqual(None, fwds)
+        self.assertEqual(0, result['exit_code'])
+        self.assertIn('result.sarif', output_files)
+
 
 class ReportFailureTest(unittest.TestCase):
 
     def assertUnderFailures(self, path):
         self.assertEqual('failures', os.path.basename(os.path.dirname(path)))
 
     def test_report_failure_create_files(self):
         with libear.TemporaryDirectory() as tmpdir: