Page MenuHomePhabricator

Introduce a CallGraph updater helper class
Needs ReviewPublic

Authored by jdoerfert on Dec 2 2019, 2:47 PM.

Details

Summary

The CallGraphUpdater is a helper that simplifies the process of updating
the call graph, both old and new style, while running an CG-SCC pass.

The uses are, for now, contained in different commits, e.g. D70767.

More functionality, e.g., replace a function with a new version, is
added as we need it.

Diff Detail

Unit TestsFailed

TimeTest
360 msLLVM-Unit.Analysis/_/AnalysisTests::CGSCCPassManagerTest.TestUpdateCGAndAnalysisManagerForPasses8
Note: Google Test filter = CGSCCPassManagerTest.TestUpdateCGAndAnalysisManagerForPasses8 [==========] Running 1 test from 1 test case. [----------] Global test environment set-up.
780 mslibc++.std/language_support/cmp/cmp_partialord::partialord.pass.cpp
Command: ['/usr/bin/clang++', '-o', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/std/language.support/cmp/cmp.partialord/Output/partialord.pass.cpp.o', '-x', 'c++', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/language.support/cmp/cmp.partialord/partialord.pass.cpp', '-c', '-v', '-ftemplate-depth=270', '-Werror=thread-safety', '-std=c++2a', '-include', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/include', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/include/c++build', '-D__STDC_FORMAT_MACROS', '-D__STDC_LIMIT_MACROS', '-D__STDC_CONSTANT_MACROS', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/input.output/filesystems/Inputs/static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/filesystem/Output/dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="/usr/bin/python /mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/filesystem_dynamic_test_helper.py"', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wall', '-Wextra', '-Werror', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-c'] Exit Code: 1 Standard Error:
800 mslibc++.std/language_support/cmp/cmp_strongeq::cmp.strongeq.pass.cpp
Command: ['/usr/bin/clang++', '-o', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/std/language.support/cmp/cmp.strongeq/Output/cmp.strongeq.pass.cpp.o', '-x', 'c++', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/language.support/cmp/cmp.strongeq/cmp.strongeq.pass.cpp', '-c', '-v', '-ftemplate-depth=270', '-Werror=thread-safety', '-std=c++2a', '-include', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/include', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/include/c++build', '-D__STDC_FORMAT_MACROS', '-D__STDC_LIMIT_MACROS', '-D__STDC_CONSTANT_MACROS', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/input.output/filesystems/Inputs/static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/filesystem/Output/dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="/usr/bin/python /mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/filesystem_dynamic_test_helper.py"', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wall', '-Wextra', '-Werror', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-c'] Exit Code: 1 Standard Error:
830 mslibc++.std/language_support/cmp/cmp_strongord::strongord.pass.cpp
Command: ['/usr/bin/clang++', '-o', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/std/language.support/cmp/cmp.strongord/Output/strongord.pass.cpp.o', '-x', 'c++', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/language.support/cmp/cmp.strongord/strongord.pass.cpp', '-c', '-v', '-ftemplate-depth=270', '-Werror=thread-safety', '-std=c++2a', '-include', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/include', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/include/c++build', '-D__STDC_FORMAT_MACROS', '-D__STDC_LIMIT_MACROS', '-D__STDC_CONSTANT_MACROS', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/input.output/filesystems/Inputs/static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/filesystem/Output/dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="/usr/bin/python /mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/filesystem_dynamic_test_helper.py"', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wall', '-Wextra', '-Werror', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-c'] Exit Code: 1 Standard Error:
780 mslibc++.std/language_support/cmp/cmp_weakeq::cmp.weakeq.pass.cpp
Command: ['/usr/bin/clang++', '-o', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/std/language.support/cmp/cmp.weakeq/Output/cmp.weakeq.pass.cpp.o', '-x', 'c++', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/language.support/cmp/cmp.weakeq/cmp.weakeq.pass.cpp', '-c', '-v', '-ftemplate-depth=270', '-Werror=thread-safety', '-std=c++2a', '-include', '/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/include', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/include/c++build', '-D__STDC_FORMAT_MACROS', '-D__STDC_LIMIT_MACROS', '-D__STDC_CONSTANT_MACROS', '-I/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support', '-DLIBCXX_FILESYSTEM_STATIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/std/input.output/filesystems/Inputs/static_test_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT="/mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/build/projects/libcxx/test/filesystem/Output/dynamic_env"', '-DLIBCXX_FILESYSTEM_DYNAMIC_TEST_HELPER="/usr/bin/python /mnt/disks/ssd0/agent/workspace/amd64_debian_testing_clang8/libcxx/test/support/filesystem_dynamic_test_helper.py"', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wall', '-Wextra', '-Werror', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-c'] Exit Code: 1 Standard Error:
View Full Test Results (7 Failed)

Event Timeline

jdoerfert created this revision.Dec 2 2019, 2:47 PM
Herald added a project: Restricted Project. · View Herald TranscriptDec 2 2019, 2:47 PM

Build result: pass - 60395 tests passed, 0 failed and 726 were skipped.

Log files: console-log.txt, CMakeCache.txt

jdoerfert updated this revision to Diff 232584.Dec 6 2019, 9:11 AM

Add comment

Build result: pass - 60558 tests passed, 0 failed and 726 were skipped.

Log files: console-log.txt, CMakeCache.txt

@hfinkel ping, this is a dependence for 2 CG-SCC patches

hfinkel added inline comments.Dec 27 2019, 12:22 AM
llvm/lib/Analysis/CGSCCPassManager.cpp
456 ↗(On Diff #232584)

This patch seems to be doing two things. One, is fixing this FIXME, and the second is adding the CallGraphUpdater wrapper. Can you split out these two changes?

Also, can you add unit tests for these various things?

llvm/lib/Transforms/Utils/CallGraphUpdater.cpp
46

What's the state of this? Should we have an assert for now?

jdoerfert updated this revision to Diff 235666.Mon, Dec 30, 7:19 PM
jdoerfert marked an inline comment as done.

Extract parts into D72025, improve the existing functionality, add unit tests.

Unit tests: pass. 61163 tests passed, 0 failed and 728 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Thanks for this! I'm depending on this patch in D71899.

I noticed an edge case when using the CallGraphUpdater to introduce coroutine funclets for recursive coroutine functions, specifically when compiling this: https://github.com/lewissbaker/cppcoro/blob/d83ee9352ea2652fbc4f02885f9d19770f5e9608/test/recursive_generator_tests.cpp#L164. I reduced this to a small unit test, and what I think may be a reasonable fix, in D72226. Please feel free to review my patch, merge it into this one, or to address the issue however you like. And please let me know if it's actually just a user error on my part.

I had a question about where registerOutlinedFunction belongs, but other than that this LGTM! I'm not an expert, though, so maybe @chandlerc or someone could review this.

llvm/include/llvm/Transforms/Utils/CallGraphUpdater.h
27

Because it's a friend of LazyCallGraph and so can modify its internal state, I'd say CallGraphUpdater is a lot more than just a wrapper to unify the two interfaces. I rely on CallGraphUpdater in my patch D71899 because it allows me to insert an outlined function node into the call graph via CallGraphUpdater:: registerOutlinedFunction, defined below.

This makes me wonder whether the functionality implemented in the body of CallGraphUpdater::registerOutlinedFunction should actually be moved, to a public member function on LazyCallGraph or one of its other helper classes. The body of that function reaches into internal mappings in LazyCallGraph and modifies them -- that seems like something only LazyCallGraph itself should be responsible for.

The other member functions of CallGraphUpdater only use CallGraph and LazyCallGraph API that are public, so those do seem like "wrappers" to me. But CallGraphUpdater::registerOutlinedFunction seems like something different. If that function is going to remain, I'd suggest updating this docblock to indicate that CallGraphUpdater provides an interface that isn't available on LazyCallGraph itself.

44

Just echoing my nit-pick comment on D72025, but I think "CGSCC" is the canonical abbreviation. "CG-SCC" only appears in this diff and D72025.

wenlei added a subscriber: wenlei.Sun, Jan 5, 10:12 AM
jdoerfert updated this revision to Diff 236474.Mon, Jan 6, 3:26 PM

More tests, late function removal and other improvements

Unit tests: pass. 61278 tests passed, 0 failed and 736 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Friendly ping for review on this one. @jdoerfert is there anything blocking this besides code review? I think the clang-tidy errors are legitimate, FWIW, but they shouldn't be blockers for code review.

Reviewers: this is a blocker for @jdoerfert's patch for an attributor CGSCC pass, as well as for a series of patches supporting coroutine passes in the new pass manager.

jdoerfert updated this revision to Diff 240027.Thu, Jan 23, 3:23 PM

Improvements after running the Attributor SCC pass on the LLVM test suite

Unit tests: fail. 62156 tests passed, 7 failed and 811 were skipped.

failed: LLVM-Unit.Analysis/_/AnalysisTests/CGSCCPassManagerTest.TestUpdateCGAndAnalysisManagerForPasses8
failed: libc++.std/language_support/cmp/cmp_partialord/partialord.pass.cpp
failed: libc++.std/language_support/cmp/cmp_strongeq/cmp.strongeq.pass.cpp
failed: libc++.std/language_support/cmp/cmp_strongord/strongord.pass.cpp
failed: libc++.std/language_support/cmp/cmp_weakeq/cmp.weakeq.pass.cpp
failed: libc++.std/language_support/cmp/cmp_weakord/weakord.pass.cpp
failed: libc++.std/thread/thread_mutex/thread_mutex_requirements/thread_mutex_requirements_mutex/thread_mutex_recursive/lock.pass.cpp

clang-tidy: fail. clang-tidy found 0 errors and 2 warnings. 0 of them are added as review comments below (why?).

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.