[clang-move] A prototype tool for moving class definition to new file.

[hokein]

Summary

This patch introduces a new tool which moves a specific class definition
from files (.h, .cc) to new files (.h, .cc), which mostly acts like
"Extract class defintion". In the long term, this tool should be
merged in to clang-refactoring as a subtool.

clang-move not only moves class definition, but also moves all the
forward declarations, functions defined in anonymous namespace and #include
headers to new files, to make sure the new files are compliable as much
as possible.

To move Foo from old.[h/cc] to new.[h/cc], use:

clang-move -name=Foo -old_header=old.h -old_cc=old.cc -new_header=new.h
-new_cc=new.cc old.cc

To move Foo from old.h to new.h, use:

clang-move -name=Foo -old_header=old.h -new_header=new.h old.cc

[hokein]

Minor cleanup.

[hokein]

Fix function name style.

[omtcyfz]

clang-move/ClangMove.h

25 ↗

(On Diff #70363)

FIXME?

[Eugene.Zelenko]

clang-move/ClangMove.h

33 ↗

(On Diff #70363)

Please add empty line before. Why don't use C++11 members initialization and = default?

83 ↗

(On Diff #70363)

Is explicit necessary?

106 ↗

(On Diff #70363)

Please add empty line before.

[Eugene.Zelenko]

clang-move/ClangMove.h

13 ↗

(On Diff #70363)

Isn't C++ headers should be after Clang headers?

[Eugene.Zelenko]

Probably old_source/new_source will be better, because different extensions are used for C++ files.

[ioeric]

NIce! Some initial comments.

clang-move/ClangMove.cpp

38 ↗

(On Diff #70363)

Might want to comment on how #include "old_header" is handled here?

39 ↗

(On Diff #70363)

Braces not needed.

103 ↗

(On Diff #70363)

Consider the code below to also include trailing comment.

clang::SourceLocation after_semi = clang::Lexer::findLocationAfterToken(
    D->getLocEnd, clang::tok::semi, *SM,
    result.Context->getLangOpts(),
    /*SkipTrailingWhitespaceAndNewLine=*/true);

113 ↗

(On Diff #70363)

Intuitively, it would be easier and more efficient if you construct the code as a string and then insert the code with one replacement.

119 ↗

(On Diff #70363)

FIXME: add header guard for new header file.

125 ↗

(On Diff #70363)

If two declarations in the same namespace are not consecutive in the vector (i.e. there is a decl in a different ns between them), will they be put into two canonical namespace blocks?

129 ↗

(On Diff #70363)

Is it possible to restrict all MovedDecl.Decl to NamedDecl so that we can use getQualifiedNameAsString instead of having a second implementation of retrieving namespaces.

Also how is anonymous namespace handled here?

157 ↗

(On Diff #70363)

FIXME: also consider comments for the declaration.

199 ↗

(On Diff #70363)

What about static functions?

210 ↗

(On Diff #70363)

FIXME: support out-of-line member variable initializations etc? Or is it already covered?

240 ↗

(On Diff #70363)

No braces.

Also, I am worrying if endswith is a safe way to compare files. For example, what if FileName is relative path?

254 ↗

(On Diff #70363)

nit: RemovedReplacement sounds like the replacement itself is being removed. RemoveReplacement would be better IMO.

clang-move/ClangMove.h

39 ↗

(On Diff #70363)

Should the Name be fully qualified?

52 ↗

(On Diff #70363)

Variable style. Here and some more below.

59 ↗

(On Diff #70363)

A comment for what this function does would be appreciated.

65 ↗

(On Diff #70363)

Is there any reason why you made this a reference?

clang-move/tool/ClangMoveMain.cpp

102 ↗

(On Diff #70363)

Indentation.

[hokein]

• Address review comments.

[hokein]

clang-move/ClangMove.cpp

38 ↗

(On Diff #70363)

Currently the old_header is also copied to new file.

103 ↗

(On Diff #70363)

But this code could not handle cases where the declaration definition has no semi at the end, such as "void f() {}"

125 ↗

(On Diff #70363)

Yes, only declarations which are consecutive in the vector and in the same namespace are put in one namespace block.

129 ↗

(On Diff #70363)

Yeah, getQualifiedNameAsString looks like a simpler way, but it doesn't suitable for our usage here.

The getQualifiedNameAsString includes the name of the class. For instance, a class method decl like void A::f() {} defined in global namespace, it retruns A::f which is not intended.

It handles anonymous namespace by wrapping like namespace { ... } // namespace, see the test.

199 ↗

(On Diff #70363)

Good catch, I missed it. Added.

210 ↗

(On Diff #70363)

Add support now.

240 ↗

(On Diff #70363)

The FileName is from the PPCallbacks::InclusionDirective APIs. From my experience, it seems always to be an absolute file path either I pass a relative file path or a absolute path to clang-move, but this is no guarantee in the document.

Using endswith is the most reliable way so far.

clang-move/ClangMove.h

39 ↗

(On Diff #70363)

It's not required. The name can be fully/partially qualified, e.g., ::a::b::X, a::b::X, b::X, X, which has the same behavior with hasName ast matcher.

It the given name is partially qualified, it will match all the class whose name ends with the given name.

65 ↗

(On Diff #70363)

To avoid the cost of making a copy of the structure.

83 ↗

(On Diff #70363)

The explicit keyword can prevent the copy-list-initialization like ClangMoveAction a = {spec, file_to_replacement}. I'm fine to remove it since it is trival.

clang-move/tool/ClangMoveMain.cpp

102 ↗

(On Diff #70363)

Indentation is correct here, clang-format doesn't complain anything about it.

[ioeric]

clang-move/ClangMove.cpp

103 ↗

(On Diff #70363)

Please see the comment for findLocationAfterToken.

If the token is not found or the location is inside a macro, the returned source location will be invalid.

129 ↗

(On Diff #70363)

I think it should be relatively easy to split the qualified name returned by getQualifiedNameAsString .

It handles anonymous namespace by wrapping like namespace { ... } // namespace, see the test.

I mean...wouldn't getNamespaces return something like "a::(anonymous)" for anonymous namespace? How do you handle this?

clang-move/ClangMove.h

39 ↗

(On Diff #70363)

It the given name is partially qualified, it will match all the class whose name ends with the given name.

In this case, all matched classes will be moved right? Might want to mention this in the comment.

65 ↗

(On Diff #70363)

If the copy is not that expensive, I'd just make a copy so that I don't need to worry about the life-cycle of Spec.

[hokein]

Address review comments.

[hokein]

clang-move/ClangMove.cpp

104 ↗

(On Diff #70538)

Aha, I see. I misused the findLocationAfterToken previously.

130 ↗

(On Diff #70538)

For anonymous namespace, getNamespaces just makes it an empty string.
Apart from that, we have to deal with other exceptions, for instance:

namespace a {
void A::f() {}
}

What we want is namespace a, but getQualifiedNameAsString returns a::A::f. Thus, using getQualifiedNameAsString wouldn't simplify the code at the moment.

[ioeric]

lg

unittests/clang-move/ClangMoveTests.cpp

122 ↗

(On Diff #71506)

It's fine for this patch, but I think we also want to add newlines be between moved declarations, especially if there were newlines in the original code.

[hokein]

Support fully quailified name only.

[hokein]

Update dependency.

[hokein]

Improve the way of detecting whether #include in old_header.

[sylvestre.ledru]

Maybe it could be mentioned in the release notes?

[sylvestre.ledru]

In D24243#1695052, @sylvestre.ledru wrote:

Maybe it could be mentioned in the release notes?

Forget about this comment, I have been distracted by the fact that it is now installed
https://reviews.llvm.org/D68423

clang-tools-extra/trunk/CMakeLists.txt

 add_subdirectory(clang-apply-replacements)
 add_subdirectory(clang-rename)
 add_subdirectory(clang-reorder-fields)
 add_subdirectory(modularize)
 if(CLANG_ENABLE_STATIC_ANALYZER)
 add_subdirectory(clang-tidy)
 add_subdirectory(clang-tidy-vs)
 endif()
 
 add_subdirectory(change-namespace)
 add_subdirectory(clang-query)
+add_subdirectory(clang-move)
 add_subdirectory(include-fixer)
 add_subdirectory(pp-trace)
 add_subdirectory(tool-template)
 
 # Add the common testsuite after all the tools.
 # TODO: Support tests with more granularity when features are off?
 if(CLANG_ENABLE_STATIC_ANALYZER AND CLANG_INCLUDE_TESTS)
 add_subdirectory(test)

clang-tools-extra/trunk/clang-move/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  support
+  )
+
+add_clang_library(clangMove
+  ClangMove.cpp
+
+  LINK_LIBS
+  clangAST
+  clangASTMatchers
+  clangBasic
+  clangFormat
+  clangFrontend
+  clangLex
+  clangTooling
+  clangToolingCore
+  )
+
+add_subdirectory(tool)

clang-tools-extra/trunk/clang-move/ClangMove.h

+//===-- ClangMove.h - Clang move  -----------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_MOVE_CLANGMOVE_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_MOVE_CLANGMOVE_H
+
+#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/Frontend/FrontendAction.h"
+#include "clang/Tooling/Core/Replacement.h"
+#include "clang/Tooling/Tooling.h"
+#include <map>
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace move {
+
+// FIXME: Make it support more types, e.g. function definitions.
+// Currently only support moving class definition.
+class ClangMoveTool : public ast_matchers::MatchFinder::MatchCallback {
+public:
+  // Information about the declaration being moved.
+  struct MovedDecl {
+    const clang::NamedDecl *Decl = nullptr;
+    clang::SourceManager *SM = nullptr;
+    MovedDecl() = default;
+    MovedDecl(const clang::NamedDecl *Decl, clang::SourceManager *SM)
+        : Decl(Decl), SM(SM) {}
+  };
+
+  struct MoveDefinitionSpec {
+    // A fully qualified name, e.g. "X", "a::X".
+    std::string Name;
+    std::string OldHeader;
+    std::string OldCC;
+    std::string NewHeader;
+    std::string NewCC;
+  };
+
+  ClangMoveTool(
+      const MoveDefinitionSpec &MoveSpec,
+      std::map<std::string, tooling::Replacements> &FileToReplacements);
+
+  void registerMatchers(ast_matchers::MatchFinder *Finder);
+
+  void run(const ast_matchers::MatchFinder::MatchResult &Result) override;
+
+  void onEndOfTranslationUnit() override;
+
+  // Add #includes from old.h/cc files. The FileName is where the #include
+  // comes from.
+  void addIncludes(llvm::StringRef IncludeLine, llvm::StringRef FileName);
+
+private:
+  void removeClassDefinitionInOldFiles();
+  void moveClassDefinitionToNewFiles();
+
+  MoveDefinitionSpec Spec;
+  // The Key is file path, value is the replacements being applied to the file.
+  std::map<std::string, tooling::Replacements> &FileToReplacements;
+  // All declarations (the class decl being moved, forward decls) that need to
+  // be moved/copy to the new files, saving in an AST-visited order.
+  std::vector<MovedDecl> MovedDecls;
+  // The declarations that needs to be removed in old.cc/h.
+  std::vector<MovedDecl> RemovedDecls;
+  // The #includes in old_header.h.
+  std::vector<std::string> HeaderIncludes;
+  // The #includes in old_cc.cc.
+  std::vector<std::string> CCIncludes;
+};
+
+class ClangMoveAction : public clang::ASTFrontendAction {
+public:
+  ClangMoveAction(
+      const ClangMoveTool::MoveDefinitionSpec &spec,
+      std::map<std::string, tooling::Replacements> &FileToReplacements)
+      : MoveTool(spec, FileToReplacements) {
+    MoveTool.registerMatchers(&MatchFinder);
+  }
+
+  ~ClangMoveAction() override = default;
+
+  std::unique_ptr<clang::ASTConsumer>
+  CreateASTConsumer(clang::CompilerInstance &Compiler,
+                    llvm::StringRef InFile) override;
+
+private:
+  ast_matchers::MatchFinder MatchFinder;
+  ClangMoveTool MoveTool;
+};
+
+class ClangMoveActionFactory : public tooling::FrontendActionFactory {
+public:
+  ClangMoveActionFactory(
+      const ClangMoveTool::MoveDefinitionSpec &Spec,
+      std::map<std::string, tooling::Replacements> &FileToReplacements)
+      : Spec(Spec), FileToReplacements(FileToReplacements) {}
+
+  clang::FrontendAction *create() override {
+    return new ClangMoveAction(Spec, FileToReplacements);
+  }
+
+private:
+  const ClangMoveTool::MoveDefinitionSpec &Spec;
+  std::map<std::string, tooling::Replacements> &FileToReplacements;
+};
+
+} // namespace move
+} // namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_MOVE_CLANGMOVE_H

clang-tools-extra/trunk/clang-move/ClangMove.cpp

+//===-- ClangMove.cpp - Implement ClangMove functationalities ---*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ClangMove.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
+#include "clang/Basic/SourceManager.h"
+#include "clang/Format/Format.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Lex/Lexer.h"
+#include "clang/Lex/Preprocessor.h"
+#include "clang/Rewrite/Core/Rewriter.h"
+#include "clang/Tooling/Core/Replacement.h"
+
+using namespace clang::ast_matchers;
+
+namespace clang {
+namespace move {
+namespace {
+
+// FIXME: Move to ASTMatcher.
+AST_POLYMORPHIC_MATCHER(isStatic, AST_POLYMORPHIC_SUPPORTED_TYPES(FunctionDecl,
+                                                                  VarDecl)) {
+  return Node.getStorageClass() == SC_Static;
+}
+
+class FindAllIncludes : public clang::PPCallbacks {
+public:
+  explicit FindAllIncludes(SourceManager *SM, ClangMoveTool *const MoveTool)
+      : SM(*SM), MoveTool(MoveTool) {}
+
+  void InclusionDirective(clang::SourceLocation HashLoc,
+                          const clang::Token & /*IncludeTok*/,
+                          StringRef FileName, bool IsAngled,
+                          clang::CharSourceRange /*FilenameRange*/,
+                          const clang::FileEntry * /*File*/,
+                          StringRef /*SearchPath*/, StringRef /*RelativePath*/,
+                          const clang::Module * /*Imported*/) override {
+    if (const auto *FileEntry = SM.getFileEntryForID(SM.getFileID(HashLoc))) {
+      if (IsAngled) {
+        MoveTool->addIncludes("#include <" + FileName.str() + ">\n",
+                              FileEntry->getName());
+      } else {
+        MoveTool->addIncludes("#include \"" + FileName.str() + "\"\n",
+                              FileEntry->getName());
+      }
+    }
+  }
+
+private:
+  const SourceManager &SM;
+  ClangMoveTool *const MoveTool;
+};
+
+clang::tooling::Replacement
+getReplacementInChangedCode(const clang::tooling::Replacements &Replacements,
+                            const clang::tooling::Replacement &Replacement) {
+  unsigned Start = Replacements.getShiftedCodePosition(Replacement.getOffset());
+  unsigned End = Replacements.getShiftedCodePosition(Replacement.getOffset() +
+                                                     Replacement.getLength());
+  return clang::tooling::Replacement(Replacement.getFilePath(), Start,
+                                     End - Start,
+                                     Replacement.getReplacementText());
+}
+
+void addOrMergeReplacement(const clang::tooling::Replacement &Replacement,
+                           clang::tooling::Replacements *Replacements) {
+  auto Err = Replacements->add(Replacement);
+  if (Err) {
+    llvm::consumeError(std::move(Err));
+    auto Replace = getReplacementInChangedCode(*Replacements, Replacement);
+    *Replacements = Replacements->merge(clang::tooling::Replacements(Replace));
+  }
+}
+
+bool IsInHeaderFile(const clang::SourceManager &SM, const clang::Decl *D,
+                    llvm::StringRef HeaderFile) {
+  if (HeaderFile.empty())
+    return false;
+  auto ExpansionLoc = SM.getExpansionLoc(D->getLocStart());
+  if (ExpansionLoc.isInvalid())
+    return false;
+
+  if (const auto *FE = SM.getFileEntryForID(SM.getFileID(ExpansionLoc)))
+    return llvm::StringRef(FE->getName()).endswith(HeaderFile);
+
+  return false;
+}
+
+std::vector<std::string> GetNamespaces(const clang::Decl *D) {
+  std::vector<std::string> Namespaces;
+  for (const auto *Context = D->getDeclContext(); Context;
+       Context = Context->getParent()) {
+    if (llvm::isa<clang::TranslationUnitDecl>(Context) ||
+        llvm::isa<clang::LinkageSpecDecl>(Context))
+      break;
+
+    if (const auto *ND = llvm::dyn_cast<clang::NamespaceDecl>(Context))
+      Namespaces.push_back(ND->getName().str());
+  }
+  std::reverse(Namespaces.begin(), Namespaces.end());
+  return Namespaces;
+}
+
+SourceLocation getLocForEndOfDecl(const clang::Decl *D,
+                                  const clang::SourceManager *SM) {
+  auto End = D->getLocEnd();
+  clang::SourceLocation AfterSemi = clang::Lexer::findLocationAfterToken(
+      End, clang::tok::semi, *SM, clang::LangOptions(),
+      /*SkipTrailingWhitespaceAndNewLine=*/true);
+  if (AfterSemi.isValid())
+    End = AfterSemi.getLocWithOffset(-1);
+  return End;
+}
+
+std::string getDeclarationSourceText(const clang::Decl *D,
+                                     const clang::SourceManager *SM) {
+  auto EndLoc = getLocForEndOfDecl(D, SM);
+  llvm::StringRef SourceText = clang::Lexer::getSourceText(
+      clang::CharSourceRange::getTokenRange(D->getLocStart(), EndLoc), *SM,
+      clang::LangOptions());
+  return SourceText.str() + "\n";
+}
+
+clang::tooling::Replacements
+createInsertedReplacements(const std::vector<std::string> &Includes,
+                           const std::vector<ClangMoveTool::MovedDecl> &Decls,
+                           llvm::StringRef FileName) {
+  clang::tooling::Replacements InsertedReplacements;
+
+  // Add #Includes.
+  std::string AllIncludesString;
+  // FIXME: Filter out the old_header.h and add header guard.
+  for (const auto &Include : Includes)
+    AllIncludesString += Include;
+  clang::tooling::Replacement InsertInclude(FileName, 0, 0, AllIncludesString);
+  addOrMergeReplacement(InsertInclude, &InsertedReplacements);
+
+  // Add moved class definition and its related declarations. All declarations
+  // in same namespace are grouped together.
+  std::vector<std::string> CurrentNamespaces;
+  for (const auto &MovedDecl : Decls) {
+    std::vector<std::string> DeclNamespaces = GetNamespaces(MovedDecl.Decl);
+    auto CurrentIt = CurrentNamespaces.begin();
+    auto DeclIt = DeclNamespaces.begin();
+    while (CurrentIt != CurrentNamespaces.end() &&
+           DeclIt != DeclNamespaces.end()) {
+      if (*CurrentIt != *DeclIt)
+        break;
+      ++CurrentIt;
+      ++DeclIt;
+    }
+    std::vector<std::string> NextNamespaces(CurrentNamespaces.begin(),
+                                            CurrentIt);
+    NextNamespaces.insert(NextNamespaces.end(), DeclIt, DeclNamespaces.end());
+    auto RemainingSize = CurrentNamespaces.end() - CurrentIt;
+    for (auto It = CurrentNamespaces.rbegin(); RemainingSize > 0;
+         --RemainingSize, ++It) {
+      assert(It < CurrentNamespaces.rend());
+      auto code = "} // namespace " + *It + "\n";
+      clang::tooling::Replacement InsertedReplacement(FileName, 0, 0, code);
+      addOrMergeReplacement(InsertedReplacement, &InsertedReplacements);
+    }
+    while (DeclIt != DeclNamespaces.end()) {
+      clang::tooling::Replacement InsertedReplacement(
+          FileName, 0, 0, "namespace " + *DeclIt + " {\n");
+      addOrMergeReplacement(InsertedReplacement, &InsertedReplacements);
+      ++DeclIt;
+    }
+
+    // FIXME: consider moving comments of the moved declaration.
+    clang::tooling::Replacement InsertedReplacement(
+        FileName, 0, 0, getDeclarationSourceText(MovedDecl.Decl, MovedDecl.SM));
+    addOrMergeReplacement(InsertedReplacement, &InsertedReplacements);
+
+    CurrentNamespaces = std::move(NextNamespaces);
+  }
+  std::reverse(CurrentNamespaces.begin(), CurrentNamespaces.end());
+  for (const auto &NS : CurrentNamespaces) {
+    clang::tooling::Replacement InsertedReplacement(
+        FileName, 0, 0, "} // namespace " + NS + "\n");
+    addOrMergeReplacement(InsertedReplacement, &InsertedReplacements);
+  }
+  return InsertedReplacements;
+}
+
+} // namespace
+
+std::unique_ptr<clang::ASTConsumer>
+ClangMoveAction::CreateASTConsumer(clang::CompilerInstance &Compiler,
+                                   StringRef /*InFile*/) {
+  Compiler.getPreprocessor().addPPCallbacks(llvm::make_unique<FindAllIncludes>(
+      &Compiler.getSourceManager(), &MoveTool));
+  return MatchFinder.newASTConsumer();
+}
+
+
+ClangMoveTool::ClangMoveTool(
+      const MoveDefinitionSpec &MoveSpec,
+      std::map<std::string, tooling::Replacements> &FileToReplacements)
+      : Spec(MoveSpec), FileToReplacements(FileToReplacements) {
+  Spec.Name = llvm::StringRef(Spec.Name).ltrim(':');
+}
+
+void ClangMoveTool::registerMatchers(ast_matchers::MatchFinder *Finder) {
+  std::string FullyQualifiedName = "::" + Spec.Name;
+  auto InOldHeader = isExpansionInFileMatching(Spec.OldHeader);
+  auto InOldCC = isExpansionInFileMatching(Spec.OldCC);
+  auto InOldFiles = anyOf(InOldHeader, InOldCC);
+  auto InMovedClass =
+      hasDeclContext(cxxRecordDecl(hasName(FullyQualifiedName)));
+
+  // Match moved class declarations.
+  auto MovedClass = cxxRecordDecl(
+      InOldFiles, hasName(FullyQualifiedName), isDefinition(),
+      hasDeclContext(anyOf(namespaceDecl(), translationUnitDecl())));
+  Finder->addMatcher(MovedClass.bind("moved_class"), this);
+
+  // Match moved class methods (static methods included) which are defined
+  // outside moved class declaration.
+  Finder->addMatcher(cxxMethodDecl(InOldFiles,
+                                   ofClass(hasName(FullyQualifiedName)),
+                                   isDefinition())
+                         .bind("class_method"),
+                     this);
+
+  // Match static member variable definition of the moved class.
+  Finder->addMatcher(varDecl(InMovedClass, InOldCC, isDefinition())
+                         .bind("class_static_var_decl"),
+                     this);
+
+  auto inAnonymousNamespace = hasParent(namespaceDecl(isAnonymous()));
+  // Match functions/variables definitions which are defined in anonymous
+  // namespace in old cc.
+  Finder->addMatcher(
+      namedDecl(anyOf(functionDecl(isDefinition()), varDecl(isDefinition())),
+                inAnonymousNamespace)
+          .bind("decls_in_anonymous_ns"),
+      this);
+
+  // Match static functions/variabale definitions in old cc.
+  Finder->addMatcher(
+      namedDecl(anyOf(functionDecl(isDefinition(), unless(InMovedClass),
+                                   isStatic(), InOldCC),
+                      varDecl(isDefinition(), unless(InMovedClass), isStatic(),
+                              InOldCC)))
+          .bind("static_decls"),
+      this);
+
+  // Match forward declarations in old header.
+  Finder->addMatcher(
+      cxxRecordDecl(unless(anyOf(isImplicit(), isDefinition())), InOldHeader)
+          .bind("fwd_decl"),
+      this);
+}
+
+void ClangMoveTool::run(const ast_matchers::MatchFinder::MatchResult &Result) {
+  if (const auto *CMD =
+          Result.Nodes.getNodeAs<clang::CXXMethodDecl>("class_method")) {
+    // Skip inline class methods. isInline() ast matcher doesn't ignore this
+    // case.
+    if (!CMD->isInlined()) {
+      MovedDecls.emplace_back(CMD, &Result.Context->getSourceManager());
+      RemovedDecls.push_back(MovedDecls.back());
+    }
+  } else if (const auto *VD = Result.Nodes.getNodeAs<clang::VarDecl>(
+                 "class_static_var_decl")) {
+    MovedDecls.emplace_back(VD, &Result.Context->getSourceManager());
+    RemovedDecls.push_back(MovedDecls.back());
+  } else if (const auto *class_decl =
+                 Result.Nodes.getNodeAs<clang::CXXRecordDecl>("moved_class")) {
+    MovedDecls.emplace_back(class_decl, &Result.Context->getSourceManager());
+    RemovedDecls.push_back(MovedDecls.back());
+  } else if (const auto *FWD =
+                 Result.Nodes.getNodeAs<clang::CXXRecordDecl>("fwd_decl")) {
+    // Skip all forwad declarations which appear after moved class declaration.
+    if (RemovedDecls.empty())
+      MovedDecls.emplace_back(FWD, &Result.Context->getSourceManager());
+  } else if (const auto *FD = Result.Nodes.getNodeAs<clang::NamedDecl>(
+                 "decls_in_anonymous_ns")) {
+    MovedDecls.emplace_back(FD, &Result.Context->getSourceManager());
+  } else if (const auto *ND =
+                 Result.Nodes.getNodeAs<clang::NamedDecl>("static_decls")) {
+    MovedDecls.emplace_back(ND, &Result.Context->getSourceManager());
+  }
+}
+
+void ClangMoveTool::addIncludes(llvm::StringRef IncludeLine,
+                                llvm::StringRef FileName) {
+  if (!Spec.OldHeader.empty() && FileName.endswith(Spec.OldHeader))
+    HeaderIncludes.push_back(IncludeLine.str());
+  else if (!Spec.OldCC.empty() && FileName.endswith(Spec.OldCC))
+    CCIncludes.push_back(IncludeLine.str());
+}
+
+void ClangMoveTool::removeClassDefinitionInOldFiles() {
+  for (const auto &MovedDecl : RemovedDecls) {
+    auto EndLoc = getLocForEndOfDecl(MovedDecl.Decl, MovedDecl.SM);
+    clang::tooling::Replacement RemoveReplacement(
+        *MovedDecl.SM, clang::CharSourceRange::getTokenRange(
+                           MovedDecl.Decl->getLocStart(), EndLoc),
+        "");
+    std::string FilePath = RemoveReplacement.getFilePath().str();
+    addOrMergeReplacement(RemoveReplacement, &FileToReplacements[FilePath]);
+  }
+}
+
+void ClangMoveTool::moveClassDefinitionToNewFiles() {
+  std::vector<MovedDecl> NewHeaderDecls;
+  std::vector<MovedDecl> NewCCDecls;
+  for (const auto &MovedDecl : MovedDecls) {
+    if (IsInHeaderFile(*MovedDecl.SM, MovedDecl.Decl, Spec.OldHeader))
+      NewHeaderDecls.push_back(MovedDecl);
+    else
+      NewCCDecls.push_back(MovedDecl);
+  }
+
+  if (!Spec.NewHeader.empty())
+    FileToReplacements[Spec.NewHeader] = createInsertedReplacements(
+        HeaderIncludes, NewHeaderDecls, Spec.NewHeader);
+  if (!Spec.NewCC.empty())
+    FileToReplacements[Spec.NewCC] =
+        createInsertedReplacements(CCIncludes, NewCCDecls, Spec.NewCC);
+}
+
+void ClangMoveTool::onEndOfTranslationUnit() {
+  if (RemovedDecls.empty())
+    return;
+  removeClassDefinitionInOldFiles();
+  moveClassDefinitionToNewFiles();
+}
+
+} // namespace move
+} // namespace clang

clang-tools-extra/trunk/clang-move/tool/CMakeLists.txt

+include_directories(${CMAKE_CURRENT_SOURCE_DIR}/..)
+
+add_clang_executable(clang-move
+  ClangMoveMain.cpp
+  )
+
+target_link_libraries(clang-move
+  clangAST
+  clangASTMatchers
+  clangBasic
+  clangFormat
+  clangFrontend
+  clangMove
+  clangRewrite
+  clangTooling
+  clangToolingCore
+  )

clang-tools-extra/trunk/clang-move/tool/ClangMoveMain.cpp

+//===-- ClangMoveMain.cpp - move defintion to new file ----------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ClangMove.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Rewrite/Core/Rewriter.h"
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "clang/Tooling/Refactoring.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Process.h"
+#include "llvm/Support/YAMLTraits.h"
+#include <set>
+#include <string>
+
+using namespace clang;
+using namespace llvm;
+
+namespace {
+std::error_code CreateNewFile(const llvm::Twine &path) {
+  int fd = 0;
+  if (std::error_code ec =
+          llvm::sys::fs::openFileForWrite(path, fd, llvm::sys::fs::F_Text))
+    return ec;
+
+  return llvm::sys::Process::SafelyCloseFileDescriptor(fd);
+}
+
+cl::OptionCategory ClangMoveCategory("clang-move options");
+
+cl::opt<std::string> Name("name", cl::desc("The name of class being moved."),
+                          cl::cat(ClangMoveCategory));
+
+cl::opt<std::string> OldHeader("old_header", cl::desc("Old header."),
+                               cl::cat(ClangMoveCategory));
+
+cl::opt<std::string> OldCC("old_cc", cl::desc("Old CC file."),
+                           cl::cat(ClangMoveCategory));
+
+cl::opt<std::string> NewHeader("new_header", cl::desc("New header."),
+                               cl::cat(ClangMoveCategory));
+
+cl::opt<std::string> NewCC("new_cc", cl::desc("New CC file."),
+                           cl::cat(ClangMoveCategory));
+
+cl::opt<std::string>
+    Style("style",
+          cl::desc("The style name used for reformatting. Default is \"llvm\""),
+          cl::init("llvm"), cl::cat(ClangMoveCategory));
+
+cl::opt<bool> Dump("dump_result",
+                   cl::desc("Dump results in JSON format to stdout."),
+                   cl::cat(ClangMoveCategory));
+
+} // namespace
+
+int main(int argc, const char **argv) {
+  tooling::CommonOptionsParser OptionsParser(argc, argv, ClangMoveCategory);
+  tooling::RefactoringTool Tool(OptionsParser.getCompilations(),
+                                OptionsParser.getSourcePathList());
+  move::ClangMoveTool::MoveDefinitionSpec Spec;
+  Spec.Name = Name;
+  Spec.OldHeader = OldHeader;
+  Spec.NewHeader = NewHeader;
+  Spec.OldCC = OldCC;
+  Spec.NewCC = NewCC;
+  auto Factory = llvm::make_unique<clang::move::ClangMoveActionFactory>(
+      Spec, Tool.getReplacements());
+  int CodeStatus = Tool.run(Factory.get());
+  if (CodeStatus)
+    return CodeStatus;
+
+  if (!NewCC.empty())
+    CreateNewFile(NewCC);
+  if (!NewHeader.empty())
+    CreateNewFile(NewHeader);
+
+  IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts(new DiagnosticOptions());
+  clang::TextDiagnosticPrinter DiagnosticPrinter(errs(), &*DiagOpts);
+  DiagnosticsEngine Diagnostics(
+      IntrusiveRefCntPtr<DiagnosticIDs>(new DiagnosticIDs()), &*DiagOpts,
+      &DiagnosticPrinter, false);
+  auto &FileMgr = Tool.getFiles();
+  SourceManager SM(Diagnostics, FileMgr);
+  Rewriter Rewrite(SM, LangOptions());
+
+  if (!formatAndApplyAllReplacements(Tool.getReplacements(), Rewrite, Style)) {
+    llvm::errs() << "Failed applying all replacements.\n";
+    return 1;
+  }
+
+  if (Dump) {
+    std::set<llvm::StringRef> Files;
+    for (const auto &it : Tool.getReplacements())
+      Files.insert(it.first);
+    auto WriteToJson = [&](llvm::raw_ostream &OS) {
+      OS << "[\n";
+      for (auto File : Files) {
+        OS << "  {\n";
+        OS << "    \"FilePath\": \"" << File << "\",\n";
+        const auto *Entry = FileMgr.getFile(File);
+        auto ID = SM.translateFile(Entry);
+        std::string Content;
+        llvm::raw_string_ostream ContentStream(Content);
+        Rewrite.getEditBuffer(ID).write(ContentStream);
+        OS << "    \"SourceText\": \""
+           << llvm::yaml::escape(ContentStream.str()) << "\"\n";
+        OS << "  }";
+        if (File != *(--Files.end()))
+          OS << ",\n";
+      }
+      OS << "\n]\n";
+    };
+    WriteToJson(llvm::outs());
+    return 0;
+  }
+
+  return Rewrite.overwriteChangedFiles();
+}

clang-tools-extra/trunk/unittests/CMakeLists.txt

 add_custom_target(ExtraToolsUnitTests)
 set_target_properties(ExtraToolsUnitTests PROPERTIES FOLDER "Extra Tools Unit Tests")
 
 function(add_extra_unittest test_dirname)
   add_unittest(ExtraToolsUnitTests ${test_dirname} ${ARGN})
 endfunction()
 
 add_subdirectory(change-namespace)
 add_subdirectory(clang-apply-replacements)
+add_subdirectory(clang-move)
 add_subdirectory(clang-query)
 add_subdirectory(clang-tidy)
 add_subdirectory(include-fixer)

clang-tools-extra/trunk/unittests/clang-move/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  support
+  )
+
+get_filename_component(INCLUDE_FIXER_SOURCE_DIR
+  ${CMAKE_CURRENT_SOURCE_DIR}/../../clang-move REALPATH)
+include_directories(
+  ${INCLUDE_FIXER_SOURCE_DIR}
+  )
+
+# We'd like to clang/unittests/Tooling/RewriterTestContext.h in the test.
+include_directories(${CLANG_SOURCE_DIR})
+
+add_extra_unittest(ClangMoveTests
+  ClangMoveTests.cpp
+  )
+
+target_link_libraries(ClangMoveTests
+  clangAST
+  clangASTMatchers
+  clangBasic
+  clangFormat
+  clangFrontend
+  clangMove
+  clangRewrite
+  clangTooling
+  clangToolingCore
+  )

clang-tools-extra/trunk/unittests/clang-move/ClangMoveTests.cpp

+//===-- ClangMoveTest.cpp - clang-move unit tests -------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ClangMove.h"
+#include "unittests/Tooling/RewriterTestContext.h"
+#include "clang/Format/Format.h"
+#include "clang/Frontend/FrontendActions.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Rewrite/Core/Rewriter.h"
+#include "clang/Tooling/Refactoring.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/StringRef.h"
+#include "gtest/gtest.h"
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace move {
+namespace {
+
+const char TestHeaderName[] = "foo.h";
+
+const char TestCCName[] = "foo.cc";
+
+const char TestHeader[] = "namespace a {\n"
+                          "class C1;\n"
+                          "namespace b {\n"
+                          "class Foo {\n"
+                          "public:\n"
+                          "  void f();\n"
+                          "\n"
+                          "private:\n"
+                          "  C1 *c1;\n"
+                          "  static int b;\n"
+                          "};\n"
+                          "\n"
+                          "class Foo2 {\n"
+                          "public:\n"
+                          "  int f();\n"
+                          "};\n"
+                          "} // namespace b\n"
+                          "} // namespace a\n";
+
+const char TestCC[] = "#include \"foo.h\"\n"
+                      "namespace a {\n"
+                      "namespace b {\n"
+                      "namespace {\n"
+                      "void f1() {}\n"
+                      "int kConstInt1 = 0;\n"
+                      "} // namespace\n"
+                      "\n"
+                      "static int kConstInt2 = 1;\n"
+                      "\n"
+                      "static int help() {\n"
+                      "  int a = 0;\n"
+                      "  return a;\n"
+                      "}\n"
+                      "\n"
+                      "void Foo::f() { f1(); }\n"
+                      "\n"
+                      "int Foo::b = 2;\n"
+                      "int Foo2::f() {\n"
+                      "  f1();\n"
+                      "  return 1;\n"
+                      "}\n"
+                      "} // namespace b\n"
+                      "} // namespace a\n";
+
+const char ExpectedTestHeader[] = "namespace a {\n"
+                                  "class C1;\n"
+                                  "namespace b {\n"
+                                  "\n"
+                                  "class Foo2 {\n"
+                                  "public:\n"
+                                  "  int f();\n"
+                                  "};\n"
+                                  "} // namespace b\n"
+                                  "} // namespace a\n";
+
+const char ExpectedTestCC[] = "#include \"foo.h\"\n"
+                              "namespace a {\n"
+                              "namespace b {\n"
+                              "namespace {\n"
+                              "void f1() {}\n"
+                              "int kConstInt1 = 0;\n"
+                              "} // namespace\n"
+                              "\n"
+                              "static int kConstInt2 = 1;\n"
+                              "\n"
+                              "static int help() {\n"
+                              "  int a = 0;\n"
+                              "  return a;\n"
+                              "}\n"
+                              "\n"
+                              "int Foo2::f() {\n"
+                              "  f1();\n"
+                              "  return 1;\n"
+                              "}\n"
+                              "} // namespace b\n"
+                              "} // namespace a\n";
+
+const char ExpectedNewHeader[] = "namespace a {\n"
+                                 "class C1;\n"
+                                 "namespace b {\n"
+                                 "class Foo {\n"
+                                 "public:\n"
+                                 "  void f();\n"
+                                 "\n"
+                                 "private:\n"
+                                 "  C1 *c1;\n"
+                                 "  static int b;\n"
+                                 "};\n"
+                                 "} // namespace b\n"
+                                 "} // namespace a\n";
+
+const char ExpectedNewCC[] = "#include \"foo.h\"\n"
+                             "namespace a {\n"
+                             "namespace b {\n"
+                             "namespace {\n"
+                             "void f1() {}\n"
+                             "int kConstInt1 = 0;\n"
+                             "} // namespace\n"
+                             "static int kConstInt2 = 1;\n"
+                             "static int help() {\n"
+                             "  int a = 0;\n"
+                             "  return a;\n"
+                             "}\n"
+                             "void Foo::f() { f1(); }\n"
+                             "int Foo::b = 2;\n"
+                             "} // namespace b\n"
+                             "} // namespace a\n";
+
+std::map<std::string, std::string>
+runClangMoveOnCode(const move::ClangMoveTool::MoveDefinitionSpec &Spec) {
+  clang::RewriterTestContext Context;
+
+  std::map<llvm::StringRef, clang::FileID> FileToFileID;
+  std::vector<std::pair<std::string, std::string>> FileToSourceText = {
+      {TestHeaderName, TestHeader}, {TestCCName, TestCC}};
+
+  auto CreateFiles = [&FileToSourceText, &Context, &FileToFileID](
+      llvm::StringRef Name, llvm::StringRef Code) {
+    if (!Name.empty()) {
+      FileToSourceText.emplace_back(Name, Code);
+      FileToFileID[Name] = Context.createInMemoryFile(Name, Code);
+    }
+  };
+  CreateFiles(Spec.NewCC, "");
+  CreateFiles(Spec.NewHeader, "");
+  CreateFiles(Spec.OldHeader, TestHeader);
+  CreateFiles(Spec.OldCC, TestCC);
+
+  std::map<std::string, tooling::Replacements> FileToReplacements;
+  ClangMoveTool MoveTool(Spec, FileToReplacements);
+  auto Factory = llvm::make_unique<clang::move::ClangMoveActionFactory>(
+      Spec, FileToReplacements);
+
+  tooling::runToolOnCodeWithArgs(
+      Factory->create(), TestCC, {"-std=c++11"}, TestCCName, "clang-move",
+      std::make_shared<PCHContainerOperations>(), FileToSourceText);
+  formatAndApplyAllReplacements(FileToReplacements, Context.Rewrite, "llvm");
+  // The Key is file name, value is the new code after moving the class.
+  std::map<std::string, std::string> Results;
+  for (const auto &It : FileToReplacements) {
+    StringRef FilePath = It.first;
+    Results[FilePath] = Context.getRewrittenText(FileToFileID[FilePath]);
+  }
+  return Results;
+}
+
+TEST(ClangMove, MoveHeaderAndCC) {
+  move::ClangMoveTool::MoveDefinitionSpec Spec;
+  Spec.Name = "a::b::Foo";
+  Spec.OldHeader = "foo.h";
+  Spec.OldCC = "foo.cc";
+  Spec.NewHeader = "new_foo.h";
+  Spec.NewCC = "new_foo.cc";
+  auto Results = runClangMoveOnCode(Spec);
+  EXPECT_EQ(ExpectedTestHeader, Results[Spec.OldHeader]);
+  EXPECT_EQ(ExpectedTestCC, Results[Spec.OldCC]);
+  EXPECT_EQ(ExpectedNewHeader, Results[Spec.NewHeader]);
+  EXPECT_EQ(ExpectedNewCC, Results[Spec.NewCC]);
+}
+
+TEST(ClangMove, MoveHeaderOnly) {
+  move::ClangMoveTool::MoveDefinitionSpec Spec;
+  Spec.Name = "a::b::Foo";
+  Spec.OldHeader = "foo.h";
+  Spec.NewHeader = "new_foo.h";
+  auto Results = runClangMoveOnCode(Spec);
+  EXPECT_EQ(2, Results.size());
+  EXPECT_EQ(ExpectedTestHeader, Results[Spec.OldHeader]);
+  EXPECT_EQ(ExpectedNewHeader, Results[Spec.NewHeader]);
+}
+
+TEST(ClangMove, MoveCCOnly) {
+  move::ClangMoveTool::MoveDefinitionSpec Spec;
+  Spec.Name = "a::b::Foo";
+  Spec.OldCC = "foo.cc";
+  Spec.NewCC = "new_foo.cc";
+  auto Results = runClangMoveOnCode(Spec);
+  EXPECT_EQ(2, Results.size());
+  EXPECT_EQ(ExpectedTestCC, Results[Spec.OldCC]);
+  EXPECT_EQ(ExpectedNewCC, Results[Spec.NewCC]);
+}
+
+TEST(ClangMove, MoveNonExistClass) {
+  move::ClangMoveTool::MoveDefinitionSpec Spec;
+  Spec.Name = "NonExistFoo";
+  Spec.OldHeader = "foo.h";
+  Spec.OldCC = "foo.cc";
+  Spec.NewHeader = "new_foo.h";
+  Spec.NewCC = "new_foo.cc";
+  auto Results = runClangMoveOnCode(Spec);
+  EXPECT_EQ(0, Results.size());
+}
+
+} // namespace
+} // namespce move
+} // namespace clang