This is an archive of the discontinued LLVM Phabricator instance.

Differential D150761

[NFC][Py Reformat] Reformat python files in clang and clang-tools-extra
ClosedPublic

Authored by thieta on May 17 2023, 1:59 AM.

Details

Reviewers
jhenderson
JDevlieghere
MatzeB
NoQ
njames93
jdoerfert
Commits
rGdd3c26a045c0: [NFC][Py Reformat] Reformat python files in clang and clang-tools-extra
Summary

This is an ongoing series of commits that are reformatting our
Python code.

Reformatting is done with black.

If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.

If you run into any problems, post to discourse about it and
we will try to help.

RFC Thread below:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Diff Detail

Event Timeline

thieta created this revision.May 17 2023, 1:59 AM
Herald added a reviewer: NoQ. · View Herald TranscriptMay 17 2023, 1:59 AM
Herald added a reviewer: njames93. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: PiotrZSL, kadircet, mattd and 7 others. · View Herald Transcript
thieta requested review of this revision.May 17 2023, 1:59 AM
Herald added a reviewer: jdoerfert. · View Herald TranscriptMay 17 2023, 1:59 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald Transcript
Herald added subscribers: cfe-commits, jplehr, sstefan1. · View Herald Transcript
barannikov88 added a subscriber: barannikov88.May 17 2023, 2:19 AM
barannikov88 added inline comments.
clang-tools-extra/clang-include-fixer/find-all-symbols/tool/run-find-all-symbols.py
9

This is unfortunate. The comment is now asymmetric.
I guess this is not a big deal, but maybe someone has an idea how to prettify it.

Herald added a subscriber: ormris. · View Herald TranscriptMay 17 2023, 2:19 AM
thieta added inline comments.May 17 2023, 8:07 AM
clang-tools-extra/clang-include-fixer/find-all-symbols/tool/run-find-all-symbols.py
9

I'll see if I can make it better after this diff has landed.

Harbormaster completed remote builds in B232544: Diff 522955.May 17 2023, 9:55 AM
MatzeB accepted this revision.May 17 2023, 10:56 AM

Inspected 5 random files and they looked fine as expected. Do you know if the test failures are unrealted?

LGTM with test failures resolved.

This revision is now accepted and ready to land.May 17 2023, 10:56 AM
thieta updated this revision to Diff 523680.May 19 2023, 12:40 AM

Rebase

PiotrZSL added a comment.May 19 2023, 2:24 AM

Checked clang-tidy scripts, looks +- fine.

Harbormaster completed remote builds in B233103: Diff 523680.May 19 2023, 2:49 AM
Closed by commit rGdd3c26a045c0: [NFC][Py Reformat] Reformat python files in clang and clang-tools-extra (authored by thieta). · Explain WhyMay 22 2023, 11:30 PM
This revision was automatically updated to reflect the committed changes.
thieta added a commit: rGdd3c26a045c0: [NFC][Py Reformat] Reformat python files in clang and clang-tools-extra.
Amir mentioned this in D143617: [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options.Jul 7 2023, 2:36 PM

Large Diff

This large diff affects 146 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
clang-tools-extra/
clang-include-fixer/
find-all-symbols/
tool/
171 lines
tool/
336 lines
clang-tidy/
930 lines
610 lines
tool/
485 lines
720 lines
clangd/
86 lines
quality/
158 lines
test/
24 lines
5 lines
remote-index/
175 lines
unittests/
20 lines
docs/
clang-tidy/
checks/
235 lines
144 lines
include-cleaner/
test/
Unit/
20 lines
13 lines
pseudo/
test/
Unit/
20 lines
13 lines
4 lines
test/
Unit/
21 lines
clang-tidy/
449 lines
55 lines
clang/
bindings/
python/
clang/
7 lines
1711 lines
16 lines
examples/
cindex/
82 lines
19 lines
tests/
cindex/
12 lines
135 lines
94 lines
28 lines
319 lines
38 lines
57 lines
13 lines
7 lines
11 lines
20 lines
92 lines
54 lines
15 lines
19 lines
191 lines
195 lines
44 lines
docs/
analyzer/
146 lines
191 lines
tools/
869 lines
34 lines
654 lines
79 lines
lib/
Tooling/
DumpTool/
353 lines
test/
AST/
120 lines
Analysis/
31 lines
91 lines
exploded-graph-rewriter/
20 lines
36 lines
scan-build/
22 lines
CodeGen/
663 lines
CodeGenHIP/
2 lines
CodeGenHLSL/
2 lines
CodeGenObjC/
4 lines
CodeGenObjCXX/
4 lines
Driver/
XRay/
27 lines
36 lines
51 lines
Format/
21 lines
Frontend/
2 lines
Headers/
4 lines
Index/
skip-parsed-bodies/
2 lines
Interpreter/
4 lines
LibClang/
4 lines
OpenMP/
4 lines
ParserHLSL/
2 lines
Sema/
6 lines
SemaCUDA/
6 lines
SemaHLSL/
2 lines
SemaObjCXX/
6 lines
SemaOpenCL/
6 lines
TableGen/
2 lines
Unit/
70 lines
286 lines
utils/
update_cc_test_checks/
39 lines
tools/
clang-format/
270 lines
77 lines
182 lines
clang-rename/
36 lines
include-mapping/
323 lines
453 lines
95 lines
libclang/
4 lines
scan-build-py/
lib/
libear/
155 lines
libscanbuild/
108 lines
657 lines
502 lines
93 lines
109 lines
177 lines
465 lines
49 lines
tests/
2 lines
functional/
cases/
38 lines
122 lines
24 lines
129 lines
89 lines
unit/
348 lines
73 lines
142 lines
70 lines
2 lines
784 lines
37 lines
scan-view/
share/
191 lines
456 lines
105 lines
utils/
ABITest/
1040 lines
224 lines
202 lines
CIndex/
80 lines
218 lines
TestUtils/
23 lines
analyzer/
211 lines
42 lines
470 lines
15 lines
33 lines
298 lines
11 lines
54 lines
20 lines
29 lines
919 lines
22 lines
check_cfc/
146 lines
49 lines
12 lines
170 lines
90 lines
161 lines
810 lines
270 lines
107 lines
perf-training/
837 lines
177 lines
www/
287 lines

Diff 524573

clang-tools-extra/clang-include-fixer/find-all-symbols/tool/run-find-all-symbols.py

#!/usr/bin/env python #!/usr/bin/env python
# #
#=- run-find-all-symbols.py - Parallel find-all-symbols runner -*- python -*-=# # =- run-find-all-symbols.py - Parallel find-all-symbols runner -*- python -*-=#
# #
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information. # See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
# #
#===------------------------------------------------------------------------===# # ===------------------------------------------------------------------------===#
barannikov88Unsubmitted
Not Done
ReplyInline Actions

This is unfortunate. The comment is now asymmetric.
I guess this is not a big deal, but maybe someone has an idea how to prettify it.

barannikov88: This is unfortunate. The comment is now asymmetric. I guess this is not a big deal, but maybe…
thietaAuthorUnsubmitted
Done
ReplyInline Actions

I'll see if I can make it better after this diff has landed.

thieta: I'll see if I can make it better after this diff has landed.
""" """
Parallel find-all-symbols runner Parallel find-all-symbols runner
================================ ================================
Runs find-all-symbols over all files in a compilation database. Runs find-all-symbols over all files in a compilation database.
Example invocations. Example invocations.
Show All 12 Lines
import shutil import shutil
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import threading import threading
def find_compilation_database(path): def find_compilation_database(path):
"""Adjusts the directory until a compilation database is found.""" """Adjusts the directory until a compilation database is found."""
result = './' result = "./"
while not os.path.isfile(os.path.join(result, path)): while not os.path.isfile(os.path.join(result, path)):
if os.path.realpath(result) == '/': if os.path.realpath(result) == "/":
print('Error: could not find compilation database.') print("Error: could not find compilation database.")
sys.exit(1) sys.exit(1)
result += '../' result += "../"
return os.path.realpath(result) return os.path.realpath(result)
def MergeSymbols(directory, args): def MergeSymbols(directory, args):
"""Merge all symbol files (yaml) in a given directory into a single file.""" """Merge all symbol files (yaml) in a given directory into a single file."""
invocation = [args.binary, '-merge-dir='+directory, args.saving_path] invocation = [args.binary, "-merge-dir=" + directory, args.saving_path]
subprocess.call(invocation) subprocess.call(invocation)
print('Merge is finished. Saving results in ' + args.saving_path) print("Merge is finished. Saving results in " + args.saving_path)
def run_find_all_symbols(args, tmpdir, build_path, queue): def run_find_all_symbols(args, tmpdir, build_path, queue):
"""Takes filenames out of queue and runs find-all-symbols on them.""" """Takes filenames out of queue and runs find-all-symbols on them."""
while True: while True:
name = queue.get() name = queue.get()
invocation = [args.binary, name, '-output-dir='+tmpdir, '-p='+build_path] invocation = [args.binary, name, "-output-dir=" + tmpdir, "-p=" + build_path]
sys.stdout.write(' '.join(invocation) + '\n') sys.stdout.write(" ".join(invocation) + "\n")
subprocess.call(invocation) subprocess.call(invocation)
queue.task_done() queue.task_done()
def main(): def main():
parser = argparse.ArgumentParser(description='Runs find-all-symbols over all' parser = argparse.ArgumentParser(
'files in a compilation database.') description="Runs find-all-symbols over all" "files in a compilation database."
parser.add_argument('-binary', metavar='PATH', )
default='./bin/find-all-symbols', parser.add_argument(
help='path to find-all-symbols binary') "-binary",
parser.add_argument('-j', type=int, default=0, metavar="PATH",
help='number of instances to be run in parallel.') default="./bin/find-all-symbols",
parser.add_argument('-p', dest='build_path', help="path to find-all-symbols binary",
help='path used to read a compilation database.') )
parser.add_argument('-saving-path', default='./find_all_symbols_db.yaml', parser.add_argument(
help='result saving path') "-j", type=int, default=0, help="number of instances to be run in parallel."
)
parser.add_argument(
"-p", dest="build_path", help="path used to read a compilation database."
)
parser.add_argument(
"-saving-path", default="./find_all_symbols_db.yaml", help="result saving path"
)
args = parser.parse_args() args = parser.parse_args()
db_path = 'compile_commands.json' db_path = "compile_commands.json"
if args.build_path is not None: if args.build_path is not None:
build_path = args.build_path build_path = args.build_path
else: else:
build_path = find_compilation_database(db_path) build_path = find_compilation_database(db_path)
tmpdir = tempfile.mkdtemp() tmpdir = tempfile.mkdtemp()
# Load the database and extract all files. # Load the database and extract all files.
database = json.load(open(os.path.join(build_path, db_path))) database = json.load(open(os.path.join(build_path, db_path)))
files = [entry['file'] for entry in database] files = [entry["file"] for entry in database]
# Filter out .rc files on Windows. CMake includes them for some reason. # Filter out .rc files on Windows. CMake includes them for some reason.
files = [f for f in files if not f.endswith('.rc')] files = [f for f in files if not f.endswith(".rc")]
max_task = args.j max_task = args.j
if max_task == 0: if max_task == 0:
max_task = multiprocessing.cpu_count() max_task = multiprocessing.cpu_count()
try: try:
# Spin up a bunch of tidy-launching threads. # Spin up a bunch of tidy-launching threads.
queue = Queue.Queue(max_task) queue = Queue.Queue(max_task)
for _ in range(max_task): for _ in range(max_task):
t = threading.Thread(target=run_find_all_symbols, t = threading.Thread(
args=(args, tmpdir, build_path, queue)) target=run_find_all_symbols, args=(args, tmpdir, build_path, queue)
)
t.daemon = True t.daemon = True
t.start() t.start()
# Fill the queue with files. # Fill the queue with files.
for name in files: for name in files:
queue.put(name) queue.put(name)
# Wait for all threads to be done. # Wait for all threads to be done.
queue.join() queue.join()
MergeSymbols(tmpdir, args) MergeSymbols(tmpdir, args)
except KeyboardInterrupt: except KeyboardInterrupt:
# This is a sad hack. Unfortunately subprocess goes # This is a sad hack. Unfortunately subprocess goes
# bonkers with ctrl-c and we start forking merrily. # bonkers with ctrl-c and we start forking merrily.
print('\nCtrl-C detected, goodbye.') print("\nCtrl-C detected, goodbye.")
os.kill(0, 9) os.kill(0, 9)
if __name__ == '__main__': if __name__ == "__main__":
main() main()

clang-tools-extra/clang-include-fixer/tool/clang-include-fixer.py

clang-tools-extra/clang-tidy/add_new_check.py

clang-tools-extra/clang-tidy/rename_check.py

clang-tools-extra/clang-tidy/tool/clang-tidy-diff.py

clang-tools-extra/clang-tidy/tool/run-clang-tidy.py

clang-tools-extra/clangd/TidyFastChecks.py

clang-tools-extra/clangd/quality/CompletionModelCodegen.py

clang-tools-extra/clangd/test/lit.cfg.py

clang-tools-extra/clangd/test/lit.local.cfg

clang-tools-extra/clangd/test/remote-index/pipeline_helper.py

clang-tools-extra/clangd/unittests/lit.cfg.py

clang-tools-extra/docs/clang-tidy/checks/gen-static-analyzer-docs.py

clang-tools-extra/docs/conf.py

clang-tools-extra/include-cleaner/test/Unit/lit.cfg.py

clang-tools-extra/include-cleaner/test/lit.cfg.py

clang-tools-extra/pseudo/test/Unit/lit.cfg.py

clang-tools-extra/pseudo/test/lit.cfg.py

clang-tools-extra/pseudo/test/lit.local.cfg

clang-tools-extra/test/Unit/lit.cfg.py

clang-tools-extra/test/clang-tidy/check_clang_tidy.py

clang-tools-extra/test/lit.cfg.py

clang/bindings/python/clang/__init__.py

clang/bindings/python/clang/cindex.py

clang/bindings/python/clang/enumerations.py

clang/bindings/python/examples/cindex/cindex-dump.py

clang/bindings/python/examples/cindex/cindex-includes.py

clang/bindings/python/tests/cindex/test_access_specifiers.py

clang/bindings/python/tests/cindex/test_cdb.py

clang/bindings/python/tests/cindex/test_code_completion.py

clang/bindings/python/tests/cindex/test_comment.py

clang/bindings/python/tests/cindex/test_cursor.py

clang/bindings/python/tests/cindex/test_cursor_kind.py

clang/bindings/python/tests/cindex/test_diagnostics.py

clang/bindings/python/tests/cindex/test_exception_specification_kind.py

clang/bindings/python/tests/cindex/test_file.py

clang/bindings/python/tests/cindex/test_index.py

clang/bindings/python/tests/cindex/test_linkage.py

clang/bindings/python/tests/cindex/test_location.py

clang/bindings/python/tests/cindex/test_tls_kind.py

clang/bindings/python/tests/cindex/test_token_kind.py

clang/bindings/python/tests/cindex/test_tokens.py

clang/bindings/python/tests/cindex/test_translation_unit.py

clang/bindings/python/tests/cindex/test_type.py

clang/bindings/python/tests/cindex/util.py

clang/docs/analyzer/conf.py

clang/docs/conf.py

clang/docs/tools/dump_ast_matchers.py

clang/docs/tools/dump_format_help.py

clang/docs/tools/dump_format_style.py

clang/docs/tools/generate_formatted_state.py

clang/lib/Tooling/DumpTool/generate_cxx_src_locs.py

clang/test/AST/gen_ast_dump_json_test.py

clang/test/Analysis/analyzer_test.py

clang/test/Analysis/check-analyzer-fixit.py

clang/test/Analysis/exploded-graph-rewriter/lit.local.cfg

clang/test/Analysis/lit.local.cfg

clang/test/Analysis/scan-build/lit.local.cfg

clang/test/CodeGen/builtins-nvptx-mma.py

clang/test/CodeGenHIP/lit.local.cfg

clang/test/CodeGenHLSL/lit.local.cfg

clang/test/CodeGenObjC/lit.local.cfg

clang/test/CodeGenObjCXX/lit.local.cfg

clang/test/Driver/XRay/lit.local.cfg

clang/test/Driver/ftime-trace-sections.py

clang/test/Driver/lit.local.cfg

clang/test/Format/lit.local.cfg

clang/test/Frontend/lit.local.cfg

clang/test/Headers/lit.local.cfg

clang/test/Index/skip-parsed-bodies/lit.local.cfg

clang/test/Interpreter/lit.local.cfg

clang/test/LibClang/lit.local.cfg

clang/test/OpenMP/lit.local.cfg

clang/test/ParserHLSL/lit.local.cfg

clang/test/Sema/lit.local.cfg

clang/test/SemaCUDA/lit.local.cfg

clang/test/SemaHLSL/lit.local.cfg

clang/test/SemaObjCXX/lit.local.cfg

clang/test/SemaOpenCL/lit.local.cfg

clang/test/TableGen/lit.local.cfg

clang/test/Unit/lit.cfg.py

clang/test/lit.cfg.py

clang/test/utils/update_cc_test_checks/lit.local.cfg

clang/tools/clang-format/clang-format-diff.py

clang/tools/clang-format/clang-format-sublime.py

clang/tools/clang-format/clang-format.py

clang/tools/clang-rename/clang-rename.py

clang/tools/include-mapping/cppreference_parser.py

clang/tools/include-mapping/gen_std.py

clang/tools/include-mapping/test.py

clang/tools/libclang/linker-script-to-export-list.py

clang/tools/scan-build-py/lib/libear/__init__.py

clang/tools/scan-build-py/lib/libscanbuild/__init__.py

clang/tools/scan-build-py/lib/libscanbuild/analyze.py

clang/tools/scan-build-py/lib/libscanbuild/arguments.py

clang/tools/scan-build-py/lib/libscanbuild/clang.py

clang/tools/scan-build-py/lib/libscanbuild/compilation.py

clang/tools/scan-build-py/lib/libscanbuild/intercept.py

clang/tools/scan-build-py/lib/libscanbuild/report.py

clang/tools/scan-build-py/lib/libscanbuild/shell.py

clang/tools/scan-build-py/tests/__init__.py

clang/tools/scan-build-py/tests/functional/cases/__init__.py

clang/tools/scan-build-py/tests/functional/cases/test_create_cdb.py

clang/tools/scan-build-py/tests/functional/cases/test_exec_anatomy.py

clang/tools/scan-build-py/tests/functional/cases/test_from_cdb.py

clang/tools/scan-build-py/tests/functional/cases/test_from_cmd.py

clang/tools/scan-build-py/tests/unit/test_analyze.py

clang/tools/scan-build-py/tests/unit/test_clang.py

clang/tools/scan-build-py/tests/unit/test_compilation.py

clang/tools/scan-build-py/tests/unit/test_intercept.py

clang/tools/scan-build-py/tests/unit/test_libear.py

clang/tools/scan-build-py/tests/unit/test_report.py

clang/tools/scan-build-py/tests/unit/test_shell.py

clang/tools/scan-view/share/Reporter.py

clang/tools/scan-view/share/ScanView.py

clang/tools/scan-view/share/startfile.py

clang/utils/ABITest/ABITestGen.py

clang/utils/ABITest/Enumeration.py

clang/utils/ABITest/TypeGen.py

clang/utils/CIndex/completion_logger_server.py

clang/utils/ClangDataFormat.py

clang/utils/TestUtils/deep-stack.py

clang/utils/analyzer/CmpRuns.py

clang/utils/analyzer/ProjectMap.py

clang/utils/analyzer/SATest.py

clang/utils/analyzer/SATestAdd.py

clang/utils/analyzer/SATestBenchmark.py

clang/utils/analyzer/SATestBuild.py

clang/utils/analyzer/SATestUpdateDiffs.py

clang/utils/analyzer/SATestUtils.py

clang/utils/analyzer/SumTimerInfo.py

clang/utils/analyzer/entrypoint.py

clang/utils/analyzer/exploded-graph-rewriter.py

clang/utils/bundle_resources.py

clang/utils/check_cfc/check_cfc.py

clang/utils/check_cfc/obj_diff.py

clang/utils/check_cfc/setup.py

clang/utils/check_cfc/test_check_cfc.py

clang/utils/clangdiag.py

clang/utils/convert_arm_neon.py

clang/utils/creduce-clang-crash.py

clang/utils/modfuzz.py

clang/utils/module-deps-to-rsp.py

clang/utils/perf-training/perf-helper.py

clang/utils/token-delta.py

clang/www/builtins.py