This is an archive of the discontinued LLVM Phabricator instance.

[clang-move] Only move used helper declarations.
ClosedPublic

Authored by hokein on Dec 12 2016, 6:00 AM.

Details

Summary

Instead of moving all the helper declarations blindly, this patch
implements an AST-based call graph solution to make clang-move only move used
helper decls to new.cc and remove unused decls in old.cc.

Depends on D27674.

Diff Detail

Repository
rL LLVM

Event Timeline

hokein updated this revision to Diff 81077.Dec 12 2016, 6:00 AM
hokein retitled this revision from to [clang-move] Only move used helper declarations..
hokein updated this object.
hokein added a reviewer: ioeric.
hokein added a subscriber: cfe-commits.
hokein updated this revision to Diff 81079.Dec 12 2016, 6:11 AM

Fix code style.

ioeric edited edge metadata.Dec 13 2016, 2:02 AM

First round of comments.

clang-move/ClangMove.cpp
492 ↗(On Diff #81079)

It seems that isStaticStorageClass is preventing combining matchers for functions, variables, and classes. Maybe only apply this matcher on functionDecl and varDecl below, so that helper classes can be matched with the same matcher?

495 ↗(On Diff #81079)

Why do we need to match classes separately? (Please explain in comments.)

514 ↗(On Diff #81079)

ClassDeclMatcher?

519 ↗(On Diff #81079)

Can we just restrict "dc" to be the top level or outermost decls in the first place? So that we can guarantee all declarations we are dealing with are those that we really care about? This would simplify the problem a bit I believe; otherwise, my gut feeling tells me there would be a lot of things that might go wrong.

587 ↗(On Diff #81079)

What about using declarations in non-anonymous namespaces in old cc? Do we also move those?

627 ↗(On Diff #81079)

Why is UnremovedDecls empty in this case btw?

637 ↗(On Diff #81079)

Maybe a better name for this variable. It's a bit hard to follow which decls use these helper decls since there are several sets of decls here like moved/removed/unremoved...

708 ↗(On Diff #81079)

Maybe RemovedDeclHelpers?

715 ↗(On Diff #81079)

IIUC, this condition makes sure helpers used by helpers are moved. If so, please explain this in the comment.

clang-move/ClangMove.h
93 ↗(On Diff #81079)

s/compliable/compilable/

95 ↗(On Diff #81079)

This also holds for functions. Maybe "when moving a symbol"

"all helper declarations ... used by moved symbols."

100 ↗(On Diff #81079)

Maybe "remaining helpers that are not used by non-moved symbols"?

166 ↗(On Diff #81079)

Is helper class considered here?

clang-move/UsedHelperDeclFinder.cpp
20 ↗(On Diff #81079)

This sentence doesn't parse?

22 ↗(On Diff #81079)

If we always only match outermost decls, we might be able to get rid of this. I feel that this kind of function (that traverse up AST) can easily hit into corner cases.

107 ↗(On Diff #81079)

Doesn't llvm::DenseSet have range insertion?

clang-move/UsedHelperDeclFinder.h
22 ↗(On Diff #81079)

What's the relationship between this and the CallGraph class in clang/Analysis/CallGraph.h?

25 ↗(On Diff #81079)

What about static decls in named namespaces? I think they can also be helpers right?

28 ↗(On Diff #81079)

Are non-moved/moved declarations in old.cc considered as nodes?

49 ↗(On Diff #81079)

What does connected mean in this context? The graph is directed; does this mean reachable from D or to D?

hokein updated this revision to Diff 81413.Dec 14 2016, 10:26 AM
hokein marked 18 inline comments as done.
hokein edited edge metadata.
  • Address review comments.
  • Add more test cases.
hokein added inline comments.Dec 14 2016, 10:27 AM
clang-move/ClangMove.cpp
492 ↗(On Diff #81079)

Seems that it is hard to reuse the same matcher for functionDecl/varDecl and CXXRecordDecl since isStaticStorageClass is not available to CXXRecordDecl. So we have to match helper classes, helper functions/vars separately. Have cleaned the code to make it clearer.

587 ↗(On Diff #81079)

Those using-decls in named namespace are covered in "using_decl" (see above if statement). Have combined them together.

627 ↗(On Diff #81079)

In this case , UnremovedDeclsInOldHeader is empty, there is no way to verify unused/used helper declarations.

715 ↗(On Diff #81079)

Yes.

clang-move/ClangMove.h
166 ↗(On Diff #81079)

Yes. Have made the comment a bit clearer.

clang-move/UsedHelperDeclFinder.h
22 ↗(On Diff #81079)

There is no relationship between them. We build our own CallGraph class to meet our use cases. The CallGraph in clang/Analysis only supports function decls, and it seems hard to reuse it. The thing we reuse is the CallGraphNode.

25 ↗(On Diff #81079)

Yeah, it is already covered. Corrected the confusing comment.

49 ↗(On Diff #81079)

renamed it to getReachableNodes.

Second rounds. Will start reviewing CallGraph code next.

clang-move/ClangMove.cpp
492 ↗(On Diff #81079)

I mean merging helper classes into helperFuncOrVar (maybe need to be renamed) so that class helpers are parallel with functionDecl and varDecl here.

459 ↗(On Diff #81413)

I got the meaning here, but the name is a bit inaccurate since this also includes TranslationUnitDecl.

461 ↗(On Diff #81413)

Maybe also explain a bit on how you handle these using and alias decls here.

Are they always copied/moved regardless where they are?

493 ↗(On Diff #81413)

Thinking about this some more. Helpers might be (incorrectly) defined in non-anonymous namespaces without static qualifier. Do we want to consider these?

698 ↗(On Diff #81413)

transitively?

clang-move/UsedHelperDeclFinder.cpp
20 ↗(On Diff #81413)

s/Because.*$/We treat a class and all its members as one single node in the call graph/?

clang-move/UsedHelperDeclFinder.h
22 ↗(On Diff #81079)

So, this should be named something like reference graph instead of call graph? Call graph might be confusing for readers without context.

34 ↗(On Diff #81413)

s/interested/interesting/

45 ↗(On Diff #81413)

what is this because... associated with?

ioeric added inline comments.Dec 15 2016, 8:13 AM
clang-move/UsedHelperDeclFinder.cpp
22 ↗(On Diff #81413)

Maybe just getOutermostDecls and add FIXME for other data types.

30 ↗(On Diff #81413)

Why do you need this? And when is a function's parent a class?

103 ↗(On Diff #81413)

Do you assume that all nodes/decls in the graph are helpers?

What if some moved Decl refers to unmoved decls?

125 ↗(On Diff #81413)

I find abbreviations hurt readability in general. Maybe just FuncRef.

128 ↗(On Diff #81413)

When would DC be null? Shouldn't we assert this instead of ignoring?

132 ↗(On Diff #81413)

Shouldn't we use getCanonicalDecl instead so that we can always get the same Decl for all canonical declaration?

And if you are doing this when adding nodes, shouldn't you also do getPreviousDecl or getCanonicalDecl for getNode and getOrInsertNode etc?

145 ↗(On Diff #81413)

I'd move this check into addNodeForDecl.

clang-move/UsedHelperDeclFinder.h
59 ↗(On Diff #81413)

s/isn't existed/doesn't exist/

61 ↗(On Diff #81413)

maybe just "all members"?

64 ↗(On Diff #81413)

So, this is not adding node?

107 ↗(On Diff #81413)

Maybe add FIXME somewhere in the code for other kinds of declarations.

110 ↗(On Diff #81413)

This function is quite confusing for me. The comment indicates that this checks if D is in UsedDecls? But the implementation does more than that. I think we need a better name and comment for this.

hokein updated this revision to Diff 81741.Dec 16 2016, 5:12 AM
hokein marked 15 inline comments as done.
  • address code review comments
  • add more tests
hokein added inline comments.Dec 16 2016, 5:13 AM
clang-move/ClangMove.cpp
492 ↗(On Diff #81079)

We need to match helper classes separately because we want to reuse these two matchers (HelperFuncOrVar, HelperClasses) in finding helpers' usage below.

459 ↗(On Diff #81413)

but`TranslationUnitDecl` also means in global namespace, right?

461 ↗(On Diff #81413)

Yeah, added comment.

493 ↗(On Diff #81413)

hmm, in theory, "helpers" defined accidently in non-anonymous namespaces are not helpers as they are visible outside of the current TU. The implementation relies on the invisible property of helpers to verify whether a function/variable is a helper, so this would be hard to achieve.

clang-move/UsedHelperDeclFinder.cpp
22 ↗(On Diff #81079)

Beside construction of the graph, this function is also used in checking whether a given Declaration is used or not.

22 ↗(On Diff #81413)

The name might be too long, I have renamed it to getOutermostClassOrFunDecls (I prefer to keep the ClassOrFunDecl postfix which clearly indicates what this function exactly does without reading the comment, I'm happy to rename it in the future if needed).

Is there any types we want to support? From my understanding, it is sufficient to cover our usage now.

30 ↗(On Diff #81413)

This is for the template method in a class.

103 ↗(On Diff #81413)

The node in the graph can present moved/unmoved decls and helpers.

What if some moved Decl refers to unmoved decls?

The graph doesn't contain this information, it only contains the reference between moved/unmoved decls and helpers. So in this case, we just move the moved Decl.

clang-move/UsedHelperDeclFinder.h
22 ↗(On Diff #81079)

But the "HelperDecl" indicates our special use case. Might be "HelperDeclGraph" or "HelperDeclDependencyGraph"?

Code is almost good. I'm just still a bit confused by names.

clang-move/ClangMove.cpp
459 ↗(On Diff #81413)

Maybe, but it is not straight-forward. I think what you really want here is top-level declarations in old cc. The name does not really imply that. For example, something defined in a function in a namespace is also in that namespace.

clang-move/UsedHelperDeclFinder.cpp
30 ↗(On Diff #81413)

Wouldn't that be handled in the next iteration?

Also, if you do another getParent here, do you also need to update DC?

103 ↗(On Diff #81413)

So, IIUC, the graph can contain both non-helper decls and helper decls. In that case, why do we name it HelperDecl Graph? And this getUsedHelperDecls does not make sense either. Would be less confusing if it is just getUsedDecls.

clang-move/UsedHelperDeclFinder.h
110 ↗(On Diff #81741)

This is still a bit confusing even with the comment.

I think the confusing part is Used, which makes it hard to understand what this function does, and I think the reason why it is hard to find a good name might due to the wrong abstraction here. Maybe just inline this function? It is just one line after all.

22 ↗(On Diff #81079)

The problem is not with HelperDecl part. It's the CallGraph part that's a bit confusing. I think HelperDeclReferenceGraph might be better?

hokein updated this revision to Diff 81762.Dec 16 2016, 8:57 AM
hokein marked 3 inline comments as done.

refactoring the patch.

Thanks for the awesome suggestions on the names. I refactored the patch, hope it is clearer now.

clang-move/UsedHelperDeclFinder.cpp
30 ↗(On Diff #81413)

Yep, that would be handled in next iteration, removed it.

103 ↗(On Diff #81413)

Good point. Renamed to getUsedDecls, but I still keep the Helper keyword in HelperDeclRefGraph, because "helper" indicates we use it for helpers.

hokein updated this revision to Diff 81763.Dec 16 2016, 9:00 AM

Update outdated comment.

ioeric added inline comments.Dec 19 2016, 2:43 AM
test/clang-move/move-used-helper-decls.cpp
1 ↗(On Diff #81763)

Can you also add test cases where class members are treated as the same node in the graph?

9 ↗(On Diff #81763)

It'd be really helpful if you can provide a brief comment on what you are testing for each test case.

202 ↗(On Diff #81763)

Maybe also add a test case to move all symbols in the old file.

hokein updated this revision to Diff 82804.Jan 2 2017, 7:31 AM
hokein marked 3 inline comments as done.

Add more test cases.

ioeric accepted this revision.Jan 2 2017, 7:44 AM
ioeric edited edge metadata.

Awesome! Let's ship it!

This revision is now accepted and ready to land.Jan 2 2017, 7:44 AM
This revision was automatically updated to reflect the committed changes.