[OpenMP] Begin Adding OpenMP Tool to Gather OpenMP Information

[jhuber6]

Summary

This begins to add support for a set of scripts that can be used to get information from OpenMP programs to better describe problems and eventually show the data to the user in formatted output. Right now the only support is forformatting the register and memory usage reports from ptxas and nvlink. This is simply done as a wrapper around clang and clang++.

[jdoerfert]

Do we need the ++ version, it looks the same.

Can we have a test for this in the OpenMP runtime test folder maybe?

[jhuber6]

Updating to remove some redundancy. The ++ variant is simply a link but it appears like that doesn't appear in the diff viewer.

[jhuber6]

Changing hard link to a symbolic link.

[jdoerfert]

I think this is a good start, we can improve on it as we go. Please check if we need license information in the python files, e.g., check other python files. Also, one nit below.

openmp/tools/analyzer/analyzer.py

29

Do we require _Z (here and above), isn't that only present for C++ ?

[jhuber6]

Fixing regex for C function names.

[jdoerfert]

LG

openmp/tools/analyzer/analyzer.py

+#!/usr/bin/env python
+
+import subprocess
+import os.path
+import yaml
+import io
+import re
+
+def parseKernelUsages(usageStr, usageDict):
+    demangler = 'c++filt -p'
+
+    def getKernelMem(usages):
+        match = re.search(r"([0-9]+) bytes cmem\[0\]", usages)
+        return match.group(1) if match else None
+    def getSharedMem(usages):
+        match = re.search(r"([0-9]+) bytes smem", usages)
+        return match.group(1) if match else None
+    def getRegisters(usages):
+        match = re.search(r"[Uu]sed ([0-9]+) registers", usages)
+        return match.group(1) if match else None
+    def demangle(fn):
+        expr = re.compile("__omp_offloading_[a-zA-Z0-9]*_[a-zA-Z0-9]*_(_Z.*_)_l[0-9]*$")
+        match = expr.search(fn)
+        function = match.group(1) if match else fn
+        output = subprocess.run(demangler.split(' ') + [function], check=True, stdout=subprocess.PIPE)
+        return output.stdout.decode('utf-8').strip()
+    def getLine(fn):
+        expr = re.compile("__omp_offloading_[a-zA-Z0-9]*_[a-zA-Z0-9]*_.*_l([0-9]*)$")
+        match = expr.search(fn)
+        return match.group(1) if match else 0
+
+    expr = re.compile("Function properties for \'?([a-zA-Z0-9_]*)\'?\n(.*,.*)\n")
+    for (fn, usages) in expr.findall(usageStr):
+        info = usageDict[fn] if fn in usageDict else dict()
+        info["Name"] = demangle(fn)
+        info["DebugLoc"] = {"File" : "unknown", "Line": getLine(fn), "Column" : 0}
+        info["Usage"] = {"Registers" : getRegisters(usages), "Shared" : getSharedMem(usages), "Kernel" : getKernelMem(usages)}
+        usageDict[fn] = info
+
+def getKernelUsage(stderr, fname='usage.yaml'):
+    remarks = [line for line in stderr.split('\n') if re.search(r"^remark:", line)]
+    ptxas = '\n'.join([line.split(':')[1].strip() for line in stderr.split('\n') if re.search(r"^ptxas info *:", line)])
+    nvlink = '\n'.join([line.split(':')[1].strip() for line in stderr.split('\n') if re.search(r"^nvlink info *:", line)])
+
+    if os.path.exists(fname):
+        with io.open(fname, 'r', encoding = 'utf-8') as f:
+            usage = yaml.load(f, Loader=yaml.Loader)
+    else:
+        usage = dict()
+
+    parseKernelUsages(ptxas, usage)
+    parseKernelUsages(nvlink, usage)
+
+    return usage

openmp/tools/analyzer/llvm-openmp-analyzer

+#!/usr/bin/env python
+
+"""
+A wrapper for Clang specialized for gathering information about OpenMP programs.
+Simple replace calls to clang or clang++ with llvm-openmp-analyzer to run the
+analysis passes.
+"""
+
+import argparse
+import subprocess
+import yaml # PyYaml to save and load analysis information
+import sys
+import io
+
+from analyzer import getKernelUsage
+
+desc = '''A wrapper around clang that runs OpenMP Analysis passes and gathers
+information about OpenMP programs.'''
+
+default_args = ["-fopenmp", "-Rpass=openmp-opt", "-Rpass-missed=openmp-opt", "-Rpass-analysis=openmp-opt"]
+
+def main():
+    compiler = ["clang++"] if sys.argv[0].endswith('++') else ["clang"]
+    parser = argparse.ArgumentParser(description=desc)
+    parser.add_argument('--usage-report-file',
+            metavar='filename',
+            default='usage.yaml',
+            help='Filename used for the OpenMP kernel usage reports in YAML format. "usage.yaml" by default.')
+    parser.add_argument('--no-usage-report', 
+            action='store_true',
+            default=False, 
+            help='Do not general a usage report for the OpenMP kernels.')
+    args, clang_args = parser.parse_known_args()
+
+    subprocess.run(compiler + default_args + clang_args, check=True)
+    output = subprocess.run(compiler + default_args + clang_args + ["-v"], stderr=subprocess.PIPE)
+    stderr = output.stderr.decode('utf-8')
+
+    if not args.no_usage_report:
+        usage = getKernelUsage(stderr, fname=args.usage_report_file)
+        with io.open(args.usage_report_file, 'w', encoding = 'utf-8') as f:
+            yaml.dump(usage, f)
+
+if __name__ == '__main__':
+    main()

openmp/tools/analyzer/llvm-openmp-analyzer++

+llvm-openmp-analyzer
 No newline at end of file