[flang][flang-omp-report] Add flang-omp-report summarising script

[josh.mottley.arm]

Summary

The flang plugin `flang-omp-report` takes one fortran file in and returns a
YAML report file of the input file. This becomes an issue when you want to
analyse an entire project into one final report.
The purpose of this Python script is to generate a final YAML
report from all of the files generated by `flang-omp-report`. The report can
have (currently) 2 formats; summary and log. Summary focuses on "summarizing"
all constructs and there clauses from all YAML files with a corresponding "count"
for each. Log instead combines the generated YAML files into one report in a
"cleaner" format. (Pseudo) Examples can be found for both formats at the top of
the script.

Co-Authored by: Ivan Zhechev <[e-mail]>

[josh.mottley.arm]

I will now be taking over this patch from Ivan.

[josh.mottley.arm]

Refactored code.

[awarzynski]

Hey @josh.mottley.arm , thanks for taking over! I like how you have refactoring this and split into functions - makes a lot of sense! Also, great job documenting them!

Currently this script is somewhere between a "Python Script" and a "Python Module". Take a look idiomatic usage for python scripts. Basically, in a script everything should be inside functions. Also, things inside the main block (i.e. if __name__ == "__main__") should be kept to the bare minimum. Could you update accordingly? This shouldn't be too much work.

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

59

When ruamel.yaml is not present, the script will still run all the way down here and fail with:

Traceback (most recent call last):
  File "flang/examples/flang-omp-report-plugin/yaml_summarizer.py", line 58, in <module>
    yaml = YAML()
NameError: name 'YAML' is not defined

221–225

[nit] IMHO, we don't need a function for this.

[josh.mottley.arm]

• Removed try...except check for importing ruamel.yaml. This also includes removing the yaml_module_not_found function.
• Added requirements.txt.
• Refactored code to include functions; main, parse_arguments, create_arg_parser.

[awarzynski]

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

4–5

[nit] Could you add a note explaining why one can't simply use the plugin here?

6

21

IMHO, you can safely remove This option (it is clear that it's an option ;-) ). On the other hand, it would be helpful to emphasis the difference between -l and the default behavior. See my suggestion inline.

42

You don't need the search pattern outside this method. And to me, find_yaml_files should only care about the directory in which to search and whether to do secretively. The search pattern itself that you pass to glob.iglob is just an implementation detail.

But your current approach is correct and this comment is just a nit.

51–55

This an important comment, but provides a solution a bit preemptively. And, by differentiating between the source and build dirs, it's become rather long. Also, I'm concerned that it might be quite hard to find a good solution and perhaps not worth it?

How about this:

# @TODO: Currently *all* yaml files are read - regardless of whether they have been generated with  'flang-omp-report' or not. This might result in the script reading files that it should ignore.

70–81

Wouldn't the output with -l only really differ when at least 2 input files were used?

96

119–136

Was this generated using /../llvm-project/flang/test/Examples/omp-device-constructs.f90? (like for process_log?)

163

[nit] That's clear from the function definition.

[josh.mottley.arm]

• Fixed issues reported from review comments.
• Added type hints where appropriate to arguments of functions.
• Removed the default arguments from functions where it was not neccesary, and added default arguments to parsed arguments.

[josh.mottley.arm]

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

70–81

The only difference when using 1 input file to original yaml file would be that the file path is stated only once at the top, whereas the original yaml file states the file with each construct.

119–136

My bad. It was generated using all the omp tests. Changed example code to just use /../llvm-project/flang/test/Examples/omp-device-constructs.yaml now since its a bit more interesting as an example. Also made both "process" functions use the same ".yaml" file for the example, so you can see the clear difference between the two methods.

[awarzynski]

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

33

70–81

It looks like -l will change the output in two cases:

• when only one file is read, the filename will only appear once at the top (as opposed to multiple times within the file)
• when multiple files are read, then the filename for each file is only printed once before the input for the corresponding file is dumped

Is this correct? Could you document it? Perhaps that's something that should appear in the module doc-string rather than here?

Also, I think that you could simplify this documentation by including a pseudo-example:

$ python3 yaml_summarizer.py -l file_1.yaml file_2.yaml
file_1.yaml
<OMP clauses and constructs from file_1.yaml>
file_2.yaml
<OMP clauses and constructs from file_2.yaml>

$ python3 yaml_summarizer.py file_1.yaml file_2.yaml
<OMP clauses and constructs from file_1.yaml and file_2.yaml>

Or something similar. This way, you make the behavior controlled with -l more obvious. Also, pasting the output for omp-device-constructs.f90 takes a lot of space, but does not really clarify much. I would just remove it. Similar comment for process_summary.

236–237

Why is this needed?

[josh.mottley.arm]

• Fixed issues reported from review comments.
• Extended '-d' argument to be able to take a list of semicolon seperated directory paths to search for '.yaml' files in.

[awarzynski]

Many thanks for working on this, LGTM!

flang/examples/flang-omp-report-plugin/requirements.txt

+ruamel.yaml==0.17.16
+ruamel.yaml.clib==0.2.6

flang/examples/flang-omp-report-plugin/yaml_summarizer.py

+"""YAML Summariser
+
+The flang plugin ``flang-omp-report`` takes one fortran
+file in and returns a YAML report file of the input file.
+This becomes an issue when you want to analyse an entire project
+into one final report.
+The purpose of this Python script is to generate a final YAML
+summary from all of the files generated by ``flang-omp-report``.
+
+Currently, it requires ``ruamel.yaml``,
+which can be installed with:
+
+    ``pip3 install ruamel.yaml``
+
+By default it scans the directory it is ran in
+for any YAML files and outputs a summary to
+stdout. It can be ran as:
+
+    ``python3 yaml_summarizer.py``
+
+Parameters:
+
+    -d   --directory   Specify which directory to scan
+
+    -l   --log         Combine all yaml files into one log (instead of generating a summary)
+
+    -o   --output      Specify a directory in which to save the summary file
+
+    -r   --recursive   Recursively search directory for all yaml files
+
+Examples:
+
+    ``python3 yaml_summarizer.py -d ~/llvm-project/build/ --r``
+
+    ``python3 yaml_summarizer.py -l -o ~/examples/report.yaml``
+"""
+
+import sys
+import glob
+import argparse
+from pathlib import Path
+from os.path import isdir
+
+from ruamel.yaml import YAML
+
+def find_yaml_files(search_directory: Path, search_pattern: str):
+    """
+    Find all '.yaml' files and returns an iglob iterator to them.
+
+    Keyword arguments:
+    search_pattern -- Search pattern for 'iglob' to use for finding '.yaml' files.
+                      If this is set to 'None', then it will default to just searching
+                      for all '.yaml' files in the current directory.
+    """
+    # @TODO: Currently *all* yaml files are read - regardless of whether they have
+    # been generated with  'flang-omp-report' or not. This might result in the script
+    # reading files that it should ignore.
+    if search_directory:
+        return glob.iglob(str(search_directory.joinpath(search_pattern)), recursive=True)
+
+    return glob.iglob(str("/" + search_pattern), recursive=True)
+
+def process_log(data, result: list):
+    """
+    Process the data input as a 'log' to the result array. This esssentially just
+    stitches together all of the input '.yaml' files into one result.
+
+    Keyword arguments:
+    data -- Data from yaml.load() for a yaml file. So the type can be 'Any'.
+    result -- Array to add the processed data to.
+
+    Example: test/Examples/omp-device-constructs.f90 after being processed to yaml:
+        /.../llvm-project/flang/test/Examples/omp-device-constructs.f90:
+        - construct: target
+          line: 18
+          clauses:
+          - clause: map
+            details: arraya
+        - construct: target
+          line: 24
+          clauses:
+          - clause: device
+            details: '0'
+        - construct: target
+          line: 30
+          clauses:
+          - clause: defaultmap
+            details: tofrom:scalar
+        - construct: teams
+          line: 36
+          clauses:
+          - clause: default
+            details: shared
+          - clause: num_teams
+            details: '3'
+          - clause: private
+            details: i
+          - clause: shared
+            details: a
+          - clause: thread_limit
+            details: '10'
+        ...
+    """
+    for datum in data:
+        items = result.get(datum['file'], [])
+        items.append({"construct" : datum['construct'],
+                        "line" : datum['line'],
+                        "clauses" : datum['clauses']})
+        result[datum['file']] = items
+
+def add_clause(datum, construct):
+    """
+    Add clauses to the construct if they're missing
+    Otherwise increment their count by one.
+
+    Keyword arguments:
+    datum -- Data construct containing clauses to check.
+    construct -- Construct to add or increment clause count.
+    """
+    to_check = [i['clause'] for i in construct['clauses']]
+    to_add = [i['clause'] for i in datum['clauses']]
+    clauses = construct["clauses"]
+    for item in to_add:
+        if item in to_check:
+            for clause in clauses:
+                if clause["clause"] == item:
+                    clause["count"] += 1
+        else:
+            clauses.append({"clause" : item,
+                            "count" : 1})
+
+def process_summary(data, result: dict):
+    """
+    Process the data input as a 'summary' to the 'result' dictionary.
+
+    Keyword arguments:
+    data -- Data from yaml.load() for a yaml file. So the type can be 'Any'.
+    result -- Dictionary to add the processed data to.
+
+    Example: test/Examples/omp-device-constructs.f90 after being processed to yaml:
+    - construct: target
+      count: 4
+      clauses:
+      - clause: map
+        count: 2
+      - clause: device
+        count: 1
+      - clause: defaultmap
+        count: 1
+    - construct: teams
+      count: 1
+      clauses:
+      - clause: default
+        count: 1
+      - clause: num_teams
+        count: 1
+      - clause: private
+        count: 1
+      - clause: shared
+        count: 1
+      - clause: thread_limit
+        count: 1
+    ...
+    """
+    for datum in data:
+        construct = next((item for item in result
+                            if item["construct"] == datum["construct"]), None)
+        clauses = []
+        # Add the construct and clauses to the summary if
+        # they haven't been seen before
+        if not construct:
+            for i in datum['clauses']:
+                clauses.append({"clause" : i['clause'],
+                                "count"    : 1})
+            result.append({"construct" : datum['construct'],
+                            "count" : 1,
+                            "clauses" : clauses})
+        else:
+            construct["count"] += 1
+
+            add_clause(datum, construct)
+
+def clean_summary(result):
+    """ Cleans the result after processing the yaml files. """
+    # Remove all "clauses" that are empty to keep things compact
+    for construct in result:
+        if construct["clauses"] == []:
+            construct.pop("clauses")
+
+def output_result(yaml: YAML, result, output_file: Path):
+    """
+    Outputs result to either 'stdout' or to a output file.
+
+    Keyword arguments:
+    result -- Format result to output.
+    output_file -- File to output result to. If this is 'None' then result will be
+                   outputted to 'stdout'.
+    """
+    if output_file:
+        with open(output_file, 'w+', encoding='utf-8') as file:
+            if output_file.suffix == ".yaml":
+                yaml.dump(result, file)
+            else:
+                file.write(result)
+    else:
+        yaml.dump(result, sys.stdout)
+
+def process_yaml(search_directory: Path, search_pattern: str,
+                 result_format: str, output_file: Path):
+    """
+    Reads each yaml file, calls the appropiate format function for
+    the file and then ouputs the result to either 'stdout' or to an output file.
+
+    Keyword arguments:
+    search_pattern -- String pattern formatted for use with glob.iglob to find all
+                      '.yaml' files.
+    result_format -- String representing output format. Current supported strings are: 'log'.
+    output_file -- Path to output file (If value is None, then default to outputting to 'stdout').
+    """
+    if result_format == "log":
+        result = {}
+        action = process_log
+        clean = None
+    else:
+        result = []
+        action = process_summary
+        clean = clean_summary
+
+    yaml = YAML()
+
+    for file in find_yaml_files(search_directory, search_pattern):
+        with open(file, "r", encoding='utf-8') as yaml_file:
+            data = yaml.load(yaml_file)
+            action(data, result)
+
+    if clean:
+        clean(result)
+
+    output_result(yaml, result, output_file)
+
+def create_arg_parser():
+    """ Create and return a argparse.ArgumentParser modified for script. """
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-d", "--directory", help="Specify a directory to scan",
+                        dest="dir", type=str)
+    parser.add_argument("-o", "--output", help="Writes to a file instead of\
+                                                stdout", dest="output", type=str)
+    parser.add_argument("-r", "--recursive", help="Recursive search for .yaml files",
+                        dest="recursive", type=bool, nargs='?', const=True, default=False)
+
+    exclusive_parser = parser.add_mutually_exclusive_group()
+    exclusive_parser.add_argument("-l", "--log", help="Modifies report format: "
+                                  "Combines the log '.yaml' files into one file.",
+                                  action='store_true', dest='log')
+    return parser
+
+def parse_arguments():
+    """ Parses arguments given to script and returns a tuple of processed arguments. """
+    parser = create_arg_parser()
+    args = parser.parse_args()
+
+    if args.dir:
+        search_directory = Path(args.dir)
+    else:
+        search_directory = Path.cwd()
+
+    if args.recursive:
+        search_pattern = "**/*.yaml"
+    else:
+        search_pattern = "*.yaml"
+
+    if args.log:
+        result_format = "log"
+    else:
+        result_format = "summary"
+
+    if args.output:
+        if isdir(args.output):
+            output_file = Path(args.output).joinpath("summary.yaml")
+        elif isdir(Path(args.output).resolve().parent):
+            output_file = Path(args.output)
+    else:
+        output_file = None
+
+    return (search_directory, search_pattern, result_format, output_file)
+
+def main():
+    """ Main function of script. """
+    (search_directory, search_pattern, result_format, output_file) = parse_arguments()
+
+    process_yaml(search_directory, search_pattern, result_format, output_file)
+
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())