[clang-format] [doc] Add script to automatically update help output in ClangFormat.rst.

[curdeius]

Summary

Fixes https://github.com/llvm/llvm-project/issues/54418.

[curdeius]

This is an alternative to D121890.
This patch goes a step further and adds a script to update the .rst file.
It isn't invoked during the build, one needs to run it manually (just as dump_format_style.py).

There's also a validation option, currently set to column width at 95 chars. I fixed some very long lines but did't try to fix everything (it would need changing help message which is used not only by clang-format).

[curdeius]

Update file comment.

[HazardyKnusperkeks]

clang/tools/clang-format/ClangFormat.cpp

105

Couldn't one split the string in python? I think an arbitrary position to split the help isn't nice. I for one have often the terminal over half the monitor spread, sometimes even the complete monitor.

[MyDeveloperDay]

Wow! Now that’s what I’m talking about!! Awesome! Thank you so much!

[curdeius]

clang/tools/clang-format/ClangFormat.cpp

105

Well, these strings are used to generate the output of clang-format --help, which is, IIUC, supposed to produce a 80-column output.
I fully agree that it would be better to have it splitted automatically depending on the output screen width, but we're not there yet unfortunately.
And .rst file is just the reflection of what --help outputs.

[sstwcw]

clang/docs/tools/dump_format_help.py

29

You intentionally did not write build/bin/clang-format to accommodate people who build in other directories, right? Sorry this comment is late.

[curdeius]

If nobody is against this patch. I'll land it soon, probably tomorrow.

clang/docs/tools/dump_format_help.py

29

I wanted to be coherent with the generate_formatted_state.py script that calls clang-format from PATH as well.
A user that builds elsewhere needs to set PATH when executing the script to use clang-format of their choice.

[MyDeveloperDay]

clang/docs/tools/dump_format_help.py

29

you definitely can't assume "build", for me my build directory is build_ninja

[sstwcw]

It looks like you forgot to chmod +x.

[owenpan]

In D121916#3440831, @sstwcw wrote:

It looks like you forgot to chmod +x.

Added in f67e3f6.

clang/docs/ClangFormat.rst

 ===========
 ClangFormat
 ===========
 
 `ClangFormat` describes a set of tools that are built on top of
 :doc:`LibFormat`. It can support your workflow in a variety of ways including a
 standalone tool and editor integrations.
 
 
 Standalone Tool
 ===============
 
 :program:`clang-format` is located in `clang/tools/clang-format` and can be used
 to format C/C++/Java/JavaScript/JSON/Objective-C/Protobuf/C# code.
 
+.. START_FORMAT_HELP
+
 .. code-block:: console
 
   $ clang-format -help
   OVERVIEW: A tool to format C/C++/Java/JavaScript/JSON/Objective-C/Protobuf/C# code.
 
   If no arguments are specified, it formats the code from standard input
   and writes the result to the standard output.
   If <file>s are given, it reformats the files. If -i is specified
   together with <file>s, the files are edited in-place. Otherwise, the
   result is written to the standard output.
 
   USAGE: clang-format [options] [<file> ...]
 
   OPTIONS:
 
   Clang-format options:
 
-    --Werror                   - If set, changes formatting warnings to errors
-    --Wno-error=<value>        - If set don't error out on the specified warning type.
-      =unknown                 -   If set, unknown format options are only warned about.
+    --Werror                       - If set, changes formatting warnings to errors
+    --Wno-error=<value>            - If set don't error out on the specified warning type.
+      =unknown                     -   If set, unknown format options are only warned about.
                                        This can be used to enable formatting, even if the
                                        configuration contains unknown (newer) options.
                                        Use with caution, as this might lead to dramatically
                                        differing format depending on an option being
                                        supported or not.
-    --assume-filename=<string> - Override filename used to determine the language.
+    --assume-filename=<string>     - Override filename used to determine the language.
                                      When reading from stdin, clang-format assumes this
                                      filename to determine the language.
-    --cursor=<uint>            - The position of the cursor when invoking
+    --cursor=<uint>                - The position of the cursor when invoking
                                      clang-format from an editor integration
-    --dry-run                  - If set, do not actually make the formatting changes
-    --dump-config              - Dump configuration options to stdout and exit.
+    --dry-run                      - If set, do not actually make the formatting changes
+    --dump-config                  - Dump configuration options to stdout and exit.
                                      Can be used with -style option.
-    --fallback-style=<string>  - The name of the predefined style used as a
+    --fallback-style=<string>      - The name of the predefined style used as a
                                      fallback in case clang-format is invoked with
                                      -style=file, but can not find the .clang-format
                                      file to use.
                                      Use -fallback-style=none to skip formatting.
-    --ferror-limit=<uint>      - Set the maximum number of clang-format errors to
-                                 emit before stopping (0 = no limit). Used only
-                                 with --dry-run or -n
-    -i                         - Inplace edit <file>s, if specified.
-    --length=<uint>            - Format a range of this length (in bytes).
+    --ferror-limit=<uint>          - Set the maximum number of clang-format errors to emit
+                                     before stopping (0 = no limit).
+                                     Used only with --dry-run or -n
+    --files=<string>               - Provide a list of files to run clang-format
+    -i                             - Inplace edit <file>s, if specified.
+    --length=<uint>                - Format a range of this length (in bytes).
                                      Multiple ranges can be formatted by specifying
                                      several -offset and -length pairs.
                                      When only a single -offset is specified without
                                      -length, clang-format will format up to the end
                                      of the file.
                                      Can only be used with one input file.
-    --lines=<string>           - <start line>:<end line> - format a range of
+    --lines=<string>               - <start line>:<end line> - format a range of
                                      lines (both 1-based).
                                      Multiple ranges can be formatted by specifying
                                      several -lines arguments.
                                      Can't be used with -offset and -length.
                                      Can only be used with one input file.
-    -n                         - Alias for --dry-run
-    --offset=<uint>            - Format a range starting at this byte offset.
+    -n                             - Alias for --dry-run
+    --offset=<uint>                - Format a range starting at this byte offset.
                                      Multiple ranges can be formatted by specifying
                                      several -offset and -length pairs.
                                      Can only be used with one input file.
-    --output-replacements-xml  - Output replacements as XML.
-    --sort-includes            - If set, overrides the include sorting behavior
+    --output-replacements-xml      - Output replacements as XML.
+    --qualifier-alignment=<string> - If set, overrides the qualifier alignment style
+                                     determined by the QualifierAlignment style flag
+    --sort-includes                - If set, overrides the include sorting behavior
                                      determined by the SortIncludes style flag
-    --style=<string>           - Coding style, currently supports:
-                                   LLVM, Google, Chromium, Mozilla, WebKit.
+    --style=<string>               - Coding style, currently supports:
+                                       LLVM, GNU, Google, Chromium, Microsoft, Mozilla, WebKit.
                                      Use -style=file to load style configuration from
                                      .clang-format file located in one of the parent
                                      directories of the source file (or current
                                      directory for stdin).
-                                 Use -style=file:<format_file_path> to load style
-                                 configuration from a format file located at
-                                 <format_file_path>. This path can be absolute or
-                                 relative to the working directory.
+                                     Use -style=file:<format_file_path> to explicitly specify
+                                     the configuration file.
                                      Use -style="{key: value, ...}" to set specific
                                      parameters, e.g.:
                                        -style="{BasedOnStyle: llvm, IndentWidth: 8}"
-    --verbose                  - If set, shows the list of processed files
+    --verbose                      - If set, shows the list of processed files
 
   Generic Options:
 
-    --help                     - Display available options (--help-hidden for more)
-    --help-list                - Display list of available options (--help-list-hidden for more)
-    --version                  - Display the version of this program
+    --help                         - Display available options (--help-hidden for more)
+    --help-list                    - Display list of available options (--help-list-hidden for more)
+    --version                      - Display the version of this program
 
 
+.. END_FORMAT_HELP
+
 When the desired code formatting style is different from the available options,
 the style can be customized using the ``-style="{key: value, ...}"`` option or
 by putting your style configuration in the ``.clang-format`` or ``_clang-format``
 file in your project's directory and using ``clang-format -style=file``.
 
 An easy way to create the ``.clang-format`` file is:
 
 .. code-block:: console

clang/docs/tools/dump_format_help.py

+#!/usr/bin/env python3
+# A tool to parse the output of `clang-format --help` and update the
+# documentation in ../ClangFormat.rst automatically.
+
+import os
+import re
+import subprocess
+import sys
+
+CLANG_DIR = os.path.join(os.path.dirname(__file__), '../..')
+DOC_FILE = os.path.join(CLANG_DIR, 'docs/ClangFormat.rst')
+
+
+def substitute(text, tag, contents):
+    replacement = '\n.. START_%s\n\n%s\n\n.. END_%s\n' % (tag, contents, tag)
+    pattern = r'\n\.\. START_%s\n.*\n\.\. END_%s\n' % (tag, tag)
+    return re.sub(pattern, '%s', text, flags=re.S) % replacement
+
+
+def indent(text, columns, indent_first_line=True):
+    indent_str = ' ' * columns
+    s = re.sub(r'\n([^\n])', '\n' + indent_str + '\\1', text, flags=re.S)
+    if not indent_first_line or s.startswith('\n'):
+        return s
+    return indent_str + s
+
+
+def get_help_output():
+    args = ["clang-format", "--help"]
+    cmd = subprocess.Popen(args, stdout=subprocess.PIPE,
+                           stderr=subprocess.STDOUT)
+    out, _ = cmd.communicate()
+    out = out.decode(sys.stdout.encoding)
+    return out
+
+
+def get_help_text():
+    out = get_help_output()
+    out = re.sub(r' clang-format\.exe ', ' clang-format ', out)
+
+    out = '''.. code-block:: console
+
+$ clang-format -help
+''' + out
+    out = indent(out, 2, indent_first_line=False)
+    return out
+
+
+def validate(text, columns):
+    for line in text.splitlines():
+        if len(line) > columns:
+            print('warning: line too long:\n', line, file=sys.stderr)
+
+
+help_text = get_help_text()
+validate(help_text, 95)
+
+with open(DOC_FILE) as f:
+    contents = f.read()
+
+contents = substitute(contents, 'FORMAT_HELP', help_text)
+
+with open(DOC_FILE, 'wb') as output:
+    output.write(contents.encode())

clang/tools/clang-format/ClangFormat.cpp

DumpConfig("dump-config",
                         "Can be used with -style option."),
                cl::cat(ClangFormatCategory));
 static cl::opt<unsigned>
     Cursor("cursor",
            cl::desc("The position of the cursor when invoking\n"
                     "clang-format from an editor integration"),
            cl::init(0), cl::cat(ClangFormatCategory));
 
-static cl::opt<bool> SortIncludes(
-    "sort-includes",
-    cl::desc("If set, overrides the include sorting behavior determined by the "
-             "SortIncludes style flag"),
+static cl::opt<bool>
+    SortIncludes("sort-includes",
+                 cl::desc("If set, overrides the include sorting behavior\n"
+                          "determined by the SortIncludes style flag"),
                  cl::cat(ClangFormatCategory));
 
 static cl::opt<std::string> QualifierAlignment(
     "qualifier-alignment",
-    cl::desc(
-        "If set, overrides the qualifier alignment style determined by the "
-        "QualifierAlignment style flag"),
+    cl::desc("If set, overrides the qualifier alignment style\n"
+             "determined by the QualifierAlignment style flag"),
     cl::init(""), cl::cat(ClangFormatCategory));
 
 static cl::opt<std::string>
     Files("files", cl::desc("Provide a list of files to run clang-format"),
           cl::init(""), cl::cat(ClangFormatCategory));
 
 static cl::opt<bool>
     Verbose("verbose", cl::desc("If set, shows the list of processed files"),

 static cl::opt<bool>
     NoWarnFormat("Wno-clang-format-violations",
                  cl::desc("Do not warn about individual formatting changes "
                           "needed. Used only with --dry-run or -n"),
                  cl::init(false), cl::cat(ClangFormatCategory), cl::Hidden);
 
 static cl::opt<unsigned> ErrorLimit(
     "ferror-limit",
-    cl::desc("Set the maximum number of clang-format errors to emit before "
-             "stopping (0 = no limit). Used only with --dry-run or -n"),
+    cl::desc("Set the maximum number of clang-format errors to emit\n"
+             "before stopping (0 = no limit).\n"
+             "Used only with --dry-run or -n"),
     cl::init(0), cl::cat(ClangFormatCategory));
 
 static cl::opt<bool>
     WarningsAsErrors("Werror",
                      cl::desc("If set, changes formatting warnings to errors"),
                      cl::cat(ClangFormatCategory));
 
 namespace {