[flang][flang-omp-report] Add flang-omp-report summarising script

[josh.mottley.arm]

Summary

The flang plugin `flang-omp-report` takes one fortran file in and returns a
YAML report file of the input file. This becomes an issue when you want to
analyse an entire project into one final report.
The purpose of this Python script is to generate a final YAML
report from all of the files generated by `flang-omp-report`. The report can
have (currently) 2 formats; summary and log. Summary focuses on "summarizing"
all constructs and there clauses from all YAML files with a corresponding "count"
for each. Log instead combines the generated YAML files into one report in a
"cleaner" format. (Pseudo) Examples can be found for both formats at the top of
the script.

Co-Authored by: Ivan Zhechev <[e-mail]>

[josh.mottley.arm]

I will now be taking over this patch from Ivan.

[josh.mottley.arm]

Refactored code.

[awarzynski]

Hey @josh.mottley.arm , thanks for taking over! I like how you have refactoring this and split into functions - makes a lot of sense! Also, great job documenting them!

Currently this script is somewhere between a "Python Script" and a "Python Module". Take a look idiomatic usage for python scripts. Basically, in a script everything should be inside functions. Also, things inside the main block (i.e. if __name__ == "__main__") should be kept to the bare minimum. Could you update accordingly? This shouldn't be too much work.

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

59

When ruamel.yaml is not present, the script will still run all the way down here and fail with:

Traceback (most recent call last):
  File "flang/examples/flang-omp-report-plugin/yaml_summarizer.py", line 58, in <module>
    yaml = YAML()
NameError: name 'YAML' is not defined

221–225

[nit] IMHO, we don't need a function for this.

[josh.mottley.arm]

• Removed try...except check for importing ruamel.yaml. This also includes removing the yaml_module_not_found function.
• Added requirements.txt.
• Refactored code to include functions; main, parse_arguments, create_arg_parser.

[awarzynski]

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

4–5

[nit] Could you add a note explaining why one can't simply use the plugin here?

6

21

IMHO, you can safely remove This option (it is clear that it's an option ;-) ). On the other hand, it would be helpful to emphasis the difference between -l and the default behavior. See my suggestion inline.

42

You don't need the search pattern outside this method. And to me, find_yaml_files should only care about the directory in which to search and whether to do secretively. The search pattern itself that you pass to glob.iglob is just an implementation detail.

But your current approach is correct and this comment is just a nit.

51–55

This an important comment, but provides a solution a bit preemptively. And, by differentiating between the source and build dirs, it's become rather long. Also, I'm concerned that it might be quite hard to find a good solution and perhaps not worth it?

How about this:

# @TODO: Currently *all* yaml files are read - regardless of whether they have been generated with  'flang-omp-report' or not. This might result in the script reading files that it should ignore.

70–81

Wouldn't the output with -l only really differ when at least 2 input files were used?

96

119–136

Was this generated using /../llvm-project/flang/test/Examples/omp-device-constructs.f90? (like for process_log?)

163

[nit] That's clear from the function definition.

[josh.mottley.arm]

• Fixed issues reported from review comments.
• Added type hints where appropriate to arguments of functions.
• Removed the default arguments from functions where it was not neccesary, and added default arguments to parsed arguments.

[josh.mottley.arm]

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

70–81

The only difference when using 1 input file to original yaml file would be that the file path is stated only once at the top, whereas the original yaml file states the file with each construct.

119–136

My bad. It was generated using all the omp tests. Changed example code to just use /../llvm-project/flang/test/Examples/omp-device-constructs.yaml now since its a bit more interesting as an example. Also made both "process" functions use the same ".yaml" file for the example, so you can see the clear difference between the two methods.

[awarzynski]

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

34

70–81

It looks like -l will change the output in two cases:

• when only one file is read, the filename will only appear once at the top (as opposed to multiple times within the file)
• when multiple files are read, then the filename for each file is only printed once before the input for the corresponding file is dumped

Is this correct? Could you document it? Perhaps that's something that should appear in the module doc-string rather than here?

Also, I think that you could simplify this documentation by including a pseudo-example:

$ python3 yaml_summarizer.py -l file_1.yaml file_2.yaml
file_1.yaml
<OMP clauses and constructs from file_1.yaml>
file_2.yaml
<OMP clauses and constructs from file_2.yaml>

$ python3 yaml_summarizer.py file_1.yaml file_2.yaml
<OMP clauses and constructs from file_1.yaml and file_2.yaml>

Or something similar. This way, you make the behavior controlled with -l more obvious. Also, pasting the output for omp-device-constructs.f90 takes a lot of space, but does not really clarify much. I would just remove it. Similar comment for process_summary.

237–238

Why is this needed?

[josh.mottley.arm]

• Fixed issues reported from review comments.
• Extended '-d' argument to be able to take a list of semicolon seperated directory paths to search for '.yaml' files in.

[awarzynski]

Many thanks for working on this, LGTM!

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

+"""YAML Summariser
+
+This Python script generates a YAML summary from
+the files generated by ``flang-omp-report``.
+Currently, it only support ``ruamel.yaml``,
+which can be installed with:
+
+    ``pip3 install ruamel.yaml``
+
+By default it scans the directory it is ran in
+for any YAML files and outputs a summary to
+stdout. It can be ran as:
+
+    ``python3 yaml_summarizer.py``
+
+Parameters:
+
+    -d   --directory   Specify which directory to scan
+
+    -l   --log         This option combines all yaml files into one
+
+    -o   --output      Specify a directory in which to save the summary file
+
+Examples:
+
+    ``python3 yaml_summarizer.py -d ~/llvm-project/build/``
+
+    ``python3 yaml_summarizer.py -l -o ~/examples/``
+"""
+
+import sys
+import glob
+import argparse
+from pathlib import Path
+
+MODULE = True
+try:
+    from ruamel.yaml import YAML
+except ImportError:
+    MODULE = False
+
+parser = argparse.ArgumentParser()
+parser.add_argument("-d", "--directory", help="Specify a directory to scan",
+                    dest="dir", type=str)
+parser.add_argument("-l", "--log", help="Combines the log files instead of\
+                                         writing out a summary", dest="log")
+parser.add_argument("-o", "--output", help="Writes to a file instead of\
+                                            stdout", dest="output", type=str)
+
+def hasYaml(path=None, log=None, output=None):
+    yaml = YAML()
+    if path:
+        files_to_scan = glob.iglob(path)
+    else:
+        files_to_scan = glob.iglob('*.yaml')
+
+    file_yaml_summary = [] # Keeps track of the total amount of constructs seen
+    file_yaml_log = dict() # Keeps track of constructs and the line they're on
+    for file in files_to_scan:
+        with open(file) as f:
+            data = yaml.load(f)
+        for datum in data:
+            items = file_yaml_log.get(datum['file'], [])
+            items.append({"construct" : datum['construct'],
+                           "line" : datum['line'],
+                           "clauses" : datum['clauses']})
+            file_yaml_log[datum['file']] = items
+
+            construct = next((item for item in file_yaml_summary
+                              if item["construct"] == datum["construct"]), None)
+            clauses = []
+            # Add the construct and clauses to the summary if
+            # they haven't been seen before
+            if not construct:
+                for i in datum['clauses']:
+                    clauses.append({"clause" : i['clause'],
+                                    "num"    : 1})
+                file_yaml_summary.append({"construct" : datum['construct'],
+                                "num" : 1,
+                                "clauses" : clauses})
+            else:
+                construct["num"] += 1
+
+                # Add clauses to the construct if they're missing
+                # Otherwise increment their count by one
+                to_check = [i['clause'] for i in construct['clauses']]
+                to_add = [i['clause'] for i in datum['clauses']]
+                clauses = construct["clauses"]
+                for item in to_add:
+                    if item in to_check:
+                        for clause in clauses:
+                            if clause["clause"] == item:
+                                clause["num"] += 1
+                    else:
+                        clauses.append({"clause" : item,
+                                        "num" : 1})
+
+    if log:
+        if output:
+            with open(output, 'w') as f:
+                yaml.dump(file_yaml_log, f)
+        else:
+            yaml.dump(file_yaml_log, sys.stdout)
+    else:
+        if output:
+            with open(output, 'w') as f:
+                yaml.dump(file_yaml_summary, f)
+        else:
+            yaml.dump(file_yaml_summary, sys.stdout)
+
+def hasNotYaml():
+    print("Currently this script only works with\
+           ``ruamel.yaml`` installed.")
+    sys.exit(1)
+
+if __name__ == "__main__":
+    if MODULE:
+        args = parser.parse_args()
+
+        p = None
+        if args.dir:
+            p = str(Path(args.dir).joinpath("*.yaml"))
+
+        l = None
+        if args.log:
+            l = True
+
+        o = None
+        if args.output:
+            o = Path(args.output).joinpath("summary.yaml")
+
+        hasYaml(p, l, o)
+    else:
+        hasNotYaml()