[libTooling] Move `RewriteRule` abstraction into its own header and impl.

[ymandel]

Summary

Move the RewriteRule class and related declarations into its own set of files
(header, implementation). Only the Transformer class is left in the
Transformer-named files. This change clarifies the distinction between the
RewriteRule class, which is essential to the Transformer library, and the
Transformer class, which is only one possible RewriteRule interpreter
(compare to TransformerClangTidyCheck, a clang-tidy based interpreter).

[gribozavr]

clang/include/clang/Tooling/Transformer/Transformer.h

58

I'm not a fan of umbrella headers... especially if they don't cover everything.

I think "include what you use" is a better rule.

The rest of the patch makes sense to me, just not this file. (OK to keep it temporarily if we need it to help clients migrate, of course.)

[ymandel]

clang/include/clang/Tooling/Transformer/Transformer.h

58

Thanks, sounds good. In that case, might it make more sense to just split off RewriteRule and not rename Transformer -> TransformerTool (along w/ the correpsonding header change)? My primary motivation for renaming the class was to allow Transformer to be the name for the umbrella.

Since Transformer.h will have to include RewriteRule.h anyhow, there won't be any need for a forwarding header while we update users.

[ymandel]

Removed umbrella-header, as per discussion.

[ymandel]

clang/include/clang/Tooling/Transformer/Transformer.h

58

I've uploaded a new diff w/ these changes. I also split the implementation file, which I'd forgotten to do previously. I've left the test in one file because testing the RewriteRule code w/o Transformer is somewhat complicated. It's probably worth doing that work, but doesn't seem critical at this point just for the sake of somewhat more independent unit tests.

[ymandel]

updated title & description to match the changes.

clang/include/clang/Tooling/Transformer/RewriteRule.h

-//===--- Transformer.h - Clang source-rewriting library ---------*- C++ -*-===//
+//===--- RewriteRule.h - RewriteRule class ----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 ///  \file
-///  Defines a library supporting the concise specification of clang-based
-///  source-to-source transformations.
+///  Defines the RewriteRule class and related functions for creating,
+///  modifying and interpreting RewriteRules.
 ///
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_CLANG_TOOLING_TRANSFORMER_TRANSFORMER_H_
-#define LLVM_CLANG_TOOLING_TRANSFORMER_TRANSFORMER_H_
+#ifndef LLVM_CLANG_TOOLING_TRANSFORMER_REWRITE_RULE_H_
+#define LLVM_CLANG_TOOLING_TRANSFORMER_REWRITE_RULE_H_
 
 #include "clang/ASTMatchers/ASTMatchFinder.h"
 #include "clang/ASTMatchers/ASTMatchers.h"
 #include "clang/ASTMatchers/ASTMatchersInternal.h"
 #include "clang/Tooling/Refactoring/AtomicChange.h"
 #include "clang/Tooling/Transformer/MatchConsumer.h"
 #include "clang/Tooling/Transformer/RangeSelector.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/Error.h"
-#include <deque>
 #include <functional>
 #include <string>
-#include <type_traits>
 #include <utility>
 
 namespace clang {
 namespace tooling {
-
 using TextGenerator = MatchConsumer<std::string>;
 
 /// Wraps a string as a TextGenerator.
 inline TextGenerator text(std::string M) {
   return [M](const ast_matchers::MatchFinder::MatchResult &)
              -> Expected<std::string> { return M; };
 }
 

 /// the match.  However, it does not fail in the case of conflicting edits --
 /// conflict handling is left to clients.  We recommend use of the \c
 /// AtomicChange or \c Replacements classes for assistance in detecting such
 /// conflicts.
 Expected<SmallVector<Transformation, 1>>
 translateEdits(const ast_matchers::MatchFinder::MatchResult &Result,
                llvm::ArrayRef<ASTEdit> Edits);
 } // namespace detail
-
-/// Handles the matcher and callback registration for a single rewrite rule, as
-/// defined by the arguments of the constructor.
-class Transformer : public ast_matchers::MatchFinder::MatchCallback {
-public:
-  using ChangeConsumer =
-      std::function<void(Expected<clang::tooling::AtomicChange> Change)>;
-
-  /// \param Consumer Receives each rewrite or error.  Will not necessarily be
-  /// called for each match; for example, if the rewrite is not applicable
-  /// because of macros, but doesn't fail.  Note that clients are responsible
-  /// for handling the case that independent \c AtomicChanges conflict with each
-  /// other.
-  Transformer(RewriteRule Rule, ChangeConsumer Consumer)
-      : Rule(std::move(Rule)), Consumer(std::move(Consumer)) {}
-
-  /// N.B. Passes `this` pointer to `MatchFinder`.  So, this object should not
-  /// be moved after this call.
-  void registerMatchers(ast_matchers::MatchFinder *MatchFinder);
-
-  /// Not called directly by users -- called by the framework, via base class
-  /// pointer.
-  void run(const ast_matchers::MatchFinder::MatchResult &Result) override;
-
-private:
-  RewriteRule Rule;
-  /// Receives each successful rewrites as an \c AtomicChange.
-  ChangeConsumer Consumer;
-};
 } // namespace tooling
 } // namespace clang
 
-#endif // LLVM_CLANG_TOOLING_TRANSFORMER_TRANSFORMER_H_
+#endif // LLVM_CLANG_TOOLING_TRANSFORMER_REWRITE_RULE_H_

clang/include/clang/Tooling/Transformer/Transformer.h

-//===--- Transformer.h - Clang source-rewriting library ---------*- C++ -*-===//
+//===--- Transformer.h - Transformer class ----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-///
-///  \file
-///  Defines a library supporting the concise specification of clang-based
-///  source-to-source transformations.
-///
-//===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLING_TRANSFORMER_TRANSFORMER_H_
 #define LLVM_CLANG_TOOLING_TRANSFORMER_TRANSFORMER_H_
 
 #include "clang/ASTMatchers/ASTMatchFinder.h"
-#include "clang/ASTMatchers/ASTMatchers.h"
-#include "clang/ASTMatchers/ASTMatchersInternal.h"
 #include "clang/Tooling/Refactoring/AtomicChange.h"
-#include "clang/Tooling/Transformer/MatchConsumer.h"
-#include "clang/Tooling/Transformer/RangeSelector.h"
-#include "llvm/ADT/STLExtras.h"
-#include "llvm/ADT/SmallVector.h"
+#include "clang/Tooling/Transformer/RewriteRule.h"
 #include "llvm/Support/Error.h"
-#include <deque>
 #include <functional>
-#include <string>
-#include <type_traits>
 #include <utility>
 
 namespace clang {
 namespace tooling {
-
-using TextGenerator = MatchConsumer<std::string>;
-
-/// Wraps a string as a TextGenerator.
-inline TextGenerator text(std::string M) {
-  return [M](const ast_matchers::MatchFinder::MatchResult &)
-             -> Expected<std::string> { return M; };
-}
-
-// Description of a source-code edit, expressed in terms of an AST node.
-// Includes: an ID for the (bound) node, a selector for source related to the
-// node, a replacement and, optionally, an explanation for the edit.
-//
-// * Target: the source code impacted by the rule. This identifies an AST node,
-//   or part thereof (\c Part), whose source range indicates the extent of the
-//   replacement applied by the replacement term.  By default, the extent is the
-//   node matched by the pattern term (\c NodePart::Node). Target's are typed
-//   (\c Kind), which guides the determination of the node extent.
-//
-// * Replacement: a function that produces a replacement string for the target,
-//   based on the match result.
-//
-// * Note: (optional) a note specifically for this edit, potentially referencing
-//   elements of the match.  This will be displayed to the user, where possible;
-//   for example, in clang-tidy diagnostics.  Use of notes should be rare --
-//   explanations of the entire rewrite should be set in the rule
-//   (`RewriteRule::Explanation`) instead.  Notes serve the rare cases wherein
-//   edit-specific diagnostics are required.
-//
-// `ASTEdit` should be built using the `change` convenience functions. For
-// example,
-// \code
-//   change(name(fun), text("Frodo"))
-// \endcode
-// Or, if we use Stencil for the TextGenerator:
-// \code
-//   using stencil::cat;
-//   change(statement(thenNode), cat("{", thenNode, "}"))
-//   change(callArgs(call), cat(x, ",", y))
-// \endcode
-// Or, if you are changing the node corresponding to the rule's matcher, you can
-// use the single-argument override of \c change:
-// \code
-//   change(cat("different_expr"))
-// \endcode
-struct ASTEdit {
-  RangeSelector TargetRange;
-  TextGenerator Replacement;
-  TextGenerator Note;
-};
-
-/// Format of the path in an include directive -- angle brackets or quotes.
-enum class IncludeFormat {
-  Quoted,
-  Angled,
-};
-
-/// Description of a source-code transformation.
-//
-// A *rewrite rule* describes a transformation of source code. A simple rule
-// contains each of the following components:
-//
-// * Matcher: the pattern term, expressed as clang matchers (with Transformer
-//   extensions).
-//
-// * Edits: a set of Edits to the source code, described with ASTEdits.
-//
-// * Explanation: explanation of the rewrite.  This will be displayed to the
-//   user, where possible; for example, in clang-tidy diagnostics.
-//
-// However, rules can also consist of (sub)rules, where the first that matches
-// is applied and the rest are ignored.  So, the above components are gathered
-// as a `Case` and a rule is a list of cases.
-//
-// Rule cases have an additional, implicit, component: the parameters. These are
-// portions of the pattern which are left unspecified, yet bound in the pattern
-// so that we can reference them in the edits.
-//
-// The \c Transformer class can be used to apply the rewrite rule and obtain the
-// corresponding replacements.
-struct RewriteRule {
-  struct Case {
-    ast_matchers::internal::DynTypedMatcher Matcher;
-    SmallVector<ASTEdit, 1> Edits;
-    TextGenerator Explanation;
-    // Include paths to add to the file affected by this case.  These are
-    // bundled with the `Case`, rather than the `RewriteRule`, because each case
-    // might have different associated changes to the includes.
-    std::vector<std::pair<std::string, IncludeFormat>> AddedIncludes;
-  };
-  // We expect RewriteRules will most commonly include only one case.
-  SmallVector<Case, 1> Cases;
-
-  // ID used as the default target of each match. The node described by the
-  // matcher is should always be bound to this id.
-  static constexpr llvm::StringLiteral RootID = "___root___";
-};
-
-/// Convenience function for constructing a simple \c RewriteRule.
-RewriteRule makeRule(ast_matchers::internal::DynTypedMatcher M,
-                     SmallVector<ASTEdit, 1> Edits,
-                     TextGenerator Explanation = nullptr);
-
-/// Convenience overload of \c makeRule for common case of only one edit.
-inline RewriteRule makeRule(ast_matchers::internal::DynTypedMatcher M,
-                            ASTEdit Edit,
-                            TextGenerator Explanation = nullptr) {
-  SmallVector<ASTEdit, 1> Edits;
-  Edits.emplace_back(std::move(Edit));
-  return makeRule(std::move(M), std::move(Edits), std::move(Explanation));
-}
-
-/// For every case in Rule, adds an include directive for the given header. The
-/// common use is assumed to be a rule with only one case. For example, to
-/// replace a function call and add headers corresponding to the new code, one
-/// could write:
-/// \code
-///   auto R = makeRule(callExpr(callee(functionDecl(hasName("foo")))),
-///            change(text("bar()")));
-///   AddInclude(R, "path/to/bar_header.h");
-///   AddInclude(R, "vector", IncludeFormat::Angled);
-/// \endcode
-void addInclude(RewriteRule &Rule, llvm::StringRef Header,
-                IncludeFormat Format = IncludeFormat::Quoted);
-
-/// Applies the first rule whose pattern matches; other rules are ignored.  If
-/// the matchers are independent then order doesn't matter. In that case,
-/// `applyFirst` is simply joining the set of rules into one.
-//
-// `applyFirst` is like an `anyOf` matcher with an edit action attached to each
-// of its cases. Anywhere you'd use `anyOf(m1.bind("id1"), m2.bind("id2"))` and
-// then dispatch on those ids in your code for control flow, `applyFirst` lifts
-// that behavior to the rule level.  So, you can write `applyFirst({makeRule(m1,
-// action1), makeRule(m2, action2), ...});`
-//
-// For example, consider a type `T` with a deterministic serialization function,
-// `serialize()`.  For performance reasons, we would like to make it
-// non-deterministic.  Therefore, we want to drop the expectation that
-// `a.serialize() = b.serialize() iff a = b` (although we'll maintain
-// `deserialize(a.serialize()) = a`).
-//
-// We have three cases to consider (for some equality function, `eq`):
-// ```
-// eq(a.serialize(), b.serialize()) --> eq(a,b)
-// eq(a, b.serialize())             --> eq(deserialize(a), b)
-// eq(a.serialize(), b)             --> eq(a, deserialize(b))
-// ```
-//
-// `applyFirst` allows us to specify each independently:
-// ```
-// auto eq_fun = functionDecl(...);
-// auto method_call = cxxMemberCallExpr(...);
-//
-// auto two_calls = callExpr(callee(eq_fun), hasArgument(0, method_call),
-//                           hasArgument(1, method_call));
-// auto left_call =
-//     callExpr(callee(eq_fun), callExpr(hasArgument(0, method_call)));
-// auto right_call =
-//     callExpr(callee(eq_fun), callExpr(hasArgument(1, method_call)));
-//
-// RewriteRule R = applyFirst({makeRule(two_calls, two_calls_action),
-//                             makeRule(left_call, left_call_action),
-//                             makeRule(right_call, right_call_action)});
-// ```
-RewriteRule applyFirst(ArrayRef<RewriteRule> Rules);
-
-/// Replaces a portion of the source text with \p Replacement.
-ASTEdit change(RangeSelector Target, TextGenerator Replacement);
-
-/// Replaces the entirety of a RewriteRule's match with \p Replacement.  For
-/// example, to replace a function call, one could write:
-/// \code
-///   makeRule(callExpr(callee(functionDecl(hasName("foo")))),
-///            change(text("bar()")))
-/// \endcode
-inline ASTEdit change(TextGenerator Replacement) {
-  return change(node(RewriteRule::RootID), std::move(Replacement));
-}
-
-/// Inserts \p Replacement before \p S, leaving the source selected by \S
-/// unchanged.
-inline ASTEdit insertBefore(RangeSelector S, TextGenerator Replacement) {
-  return change(before(std::move(S)), std::move(Replacement));
-}
-
-/// Inserts \p Replacement after \p S, leaving the source selected by \S
-/// unchanged.
-inline ASTEdit insertAfter(RangeSelector S, TextGenerator Replacement) {
-  return change(after(std::move(S)), std::move(Replacement));
-}
-
-/// Removes the source selected by \p S.
-inline ASTEdit remove(RangeSelector S) {
-  return change(std::move(S), text(""));
-}
-
-/// The following three functions are a low-level part of the RewriteRule
-/// API. We expose them for use in implementing the fixtures that interpret
-/// RewriteRule, like Transformer and TransfomerTidy, or for more advanced
-/// users.
-//
-// FIXME: These functions are really public, if advanced, elements of the
-// RewriteRule API.  Recast them as such.  Or, just declare these functions
-// public and well-supported and move them out of `detail`.
-namespace detail {
-/// Builds a single matcher for the rule, covering all of the rule's cases.
-/// Only supports Rules whose cases' matchers share the same base "kind"
-/// (`Stmt`, `Decl`, etc.)  Deprecated: use `buildMatchers` instead, which
-/// supports mixing matchers of different kinds.
-ast_matchers::internal::DynTypedMatcher buildMatcher(const RewriteRule &Rule);
-
-/// Builds a set of matchers that cover the rule (one for each distinct node
-/// matcher base kind: Stmt, Decl, etc.). Node-matchers for `QualType` and
-/// `Type` are not permitted, since such nodes carry no source location
-/// information and are therefore not relevant for rewriting. If any such
-/// matchers are included, will return an empty vector.
-std::vector<ast_matchers::internal::DynTypedMatcher>
-buildMatchers(const RewriteRule &Rule);
-
-/// Gets the beginning location of the source matched by a rewrite rule. If the
-/// match occurs within a macro expansion, returns the beginning of the
-/// expansion point. `Result` must come from the matching of a rewrite rule.
-SourceLocation
-getRuleMatchLoc(const ast_matchers::MatchFinder::MatchResult &Result);
-
-/// Returns the \c Case of \c Rule that was selected in the match result.
-/// Assumes a matcher built with \c buildMatcher.
-const RewriteRule::Case &
-findSelectedCase(const ast_matchers::MatchFinder::MatchResult &Result,
-                 const RewriteRule &Rule);
-
-/// A source "transformation," represented by a character range in the source to
-/// be replaced and a corresponding replacement string.
-struct Transformation {
-  CharSourceRange Range;
-  std::string Replacement;
-};
-
-/// Attempts to translate `Edits`, which are in terms of AST nodes bound in the
-/// match `Result`, into Transformations, which are in terms of the source code
-/// text.
-///
-/// Returns an empty vector if any of the edits apply to portions of the source
-/// that are ineligible for rewriting (certain interactions with macros, for
-/// example).  Fails if any invariants are violated relating to bound nodes in
-/// the match.  However, it does not fail in the case of conflicting edits --
-/// conflict handling is left to clients.  We recommend use of the \c
-/// AtomicChange or \c Replacements classes for assistance in detecting such
-/// conflicts.
-Expected<SmallVector<Transformation, 1>>
-translateEdits(const ast_matchers::MatchFinder::MatchResult &Result,
-               llvm::ArrayRef<ASTEdit> Edits);
-} // namespace detail
-
-/// Handles the matcher and callback registration for a single rewrite rule, as
+/// Handles the matcher and callback registration for a single `RewriteRule`, as
 /// defined by the arguments of the constructor.
 class Transformer : public ast_matchers::MatchFinder::MatchCallback {
 public:
   using ChangeConsumer =
       std::function<void(Expected<clang::tooling::AtomicChange> Change)>;
 
   /// \param Consumer Receives each rewrite or error.  Will not necessarily be
   /// called for each match; for example, if the rewrite is not applicable

 private:
   RewriteRule Rule;
   /// Receives each successful rewrites as an \c AtomicChange.
   ChangeConsumer Consumer;
 };
 } // namespace tooling
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLING_TRANSFORMER_TRANSFORMER_H_

clang/lib/Tooling/Transformer/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS Support)
 
 add_clang_library(clangTransformer
   RangeSelector.cpp
+  RewriteRule.cpp
   SourceCode.cpp
   SourceCodeBuilders.cpp
   Stencil.cpp
   Transformer.cpp
 
   LINK_LIBS
   clangAST
   clangASTMatchers
   clangBasic
   clangLex
   clangToolingCore
   clangToolingRefactoring
   )

clang/lib/Tooling/Transformer/RewriteRule.cpp

 //===--- Transformer.cpp - Transformer library implementation ---*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#include "clang/Tooling/Transformer/Transformer.h"
-#include "clang/AST/Expr.h"
+#include "clang/Tooling/Transformer/RewriteRule.h"
 #include "clang/ASTMatchers/ASTMatchFinder.h"
 #include "clang/ASTMatchers/ASTMatchers.h"
-#include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/SourceLocation.h"
-#include "clang/Rewrite/Core/Rewriter.h"
-#include "clang/Tooling/Refactoring/AtomicChange.h"
 #include "clang/Tooling/Transformer/SourceCode.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
 #include <map>
 #include <string>
 #include <utility>
 #include <vector>
 
 using namespace clang;
 using namespace tooling;
 
 using ast_matchers::MatchFinder;
 using ast_matchers::internal::DynTypedMatcher;
 using ast_type_traits::ASTNodeKind;
-using ast_type_traits::DynTypedNode;
-using llvm::Error;
-using llvm::StringError;
 
 using MatchResult = MatchFinder::MatchResult;
 
 Expected<SmallVector<tooling::detail::Transformation, 1>>
 tooling::detail::translateEdits(const MatchResult &Result,
                                 llvm::ArrayRef<ASTEdit> Edits) {
   SmallVector<tooling::detail::Transformation, 1> Transformations;
   for (const auto &Edit : Edits) {
for (size_t i = 0, N = Rule.Cases.size(); i < N; ++i) {
     std::string Tag = ("Tag" + Twine(i)).str();
     if (NodesMap.find(Tag) != NodesMap.end())
       return Rule.Cases[i];
   }
   llvm_unreachable("No tag found for this rule.");
 }
 
 constexpr llvm::StringLiteral RewriteRule::RootID;
-
-void Transformer::registerMatchers(MatchFinder *MatchFinder) {
-  for (auto &Matcher : tooling::detail::buildMatchers(Rule))
-    MatchFinder->addDynamicMatcher(Matcher, this);
-}
-
-void Transformer::run(const MatchResult &Result) {
-  if (Result.Context->getDiagnostics().hasErrorOccurred())
-    return;
-
-  RewriteRule::Case Case = tooling::detail::findSelectedCase(Result, Rule);
-  auto Transformations = tooling::detail::translateEdits(Result, Case.Edits);
-  if (!Transformations) {
-    Consumer(Transformations.takeError());
-    return;
-  }
-
-  if (Transformations->empty()) {
-    // No rewrite applied (but no error encountered either).
-    detail::getRuleMatchLoc(Result).print(
-        llvm::errs() << "note: skipping match at loc ", *Result.SourceManager);
-    llvm::errs() << "\n";
-    return;
-  }
-
-  // Record the results in the AtomicChange, anchored at the location of the
-  // first change.
-  AtomicChange AC(*Result.SourceManager,
-                  (*Transformations)[0].Range.getBegin());
-  for (const auto &T : *Transformations) {
-    if (auto Err = AC.replace(*Result.SourceManager, T.Range, T.Replacement)) {
-      Consumer(std::move(Err));
-      return;
-    }
-  }
-
-  for (const auto &I : Case.AddedIncludes) {
-    auto &Header = I.first;
-    switch (I.second) {
-    case IncludeFormat::Quoted:
-      AC.addHeader(Header);
-      break;
-    case IncludeFormat::Angled:
-      AC.addHeader((llvm::Twine("<") + Header + ">").str());
-      break;
-    }
-  }
-
-  Consumer(std::move(AC));
-}

clang/lib/Tooling/Transformer/Transformer.cpp

 //===--- Transformer.cpp - Transformer library implementation ---*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Tooling/Transformer/Transformer.h"
-#include "clang/AST/Expr.h"
 #include "clang/ASTMatchers/ASTMatchFinder.h"
-#include "clang/ASTMatchers/ASTMatchers.h"
-#include "clang/Basic/Diagnostic.h"
+#include "clang/ASTMatchers/ASTMatchersInternal.h"
 #include "clang/Basic/SourceLocation.h"
-#include "clang/Rewrite/Core/Rewriter.h"
 #include "clang/Tooling/Refactoring/AtomicChange.h"
-#include "clang/Tooling/Transformer/SourceCode.h"
-#include "llvm/ADT/Optional.h"
-#include "llvm/ADT/StringRef.h"
-#include "llvm/Support/Errc.h"
 #include "llvm/Support/Error.h"
-#include <map>
-#include <string>
 #include <utility>
 #include <vector>
 
 using namespace clang;
 using namespace tooling;
 
 using ast_matchers::MatchFinder;
-using ast_matchers::internal::DynTypedMatcher;
-using ast_type_traits::ASTNodeKind;
-using ast_type_traits::DynTypedNode;
-using llvm::Error;
-using llvm::StringError;
-
-using MatchResult = MatchFinder::MatchResult;
-
-Expected<SmallVector<tooling::detail::Transformation, 1>>
-tooling::detail::translateEdits(const MatchResult &Result,
-                                llvm::ArrayRef<ASTEdit> Edits) {
-  SmallVector<tooling::detail::Transformation, 1> Transformations;
-  for (const auto &Edit : Edits) {
-    Expected<CharSourceRange> Range = Edit.TargetRange(Result);
-    if (!Range)
-      return Range.takeError();
-    llvm::Optional<CharSourceRange> EditRange =
-        getRangeForEdit(*Range, *Result.Context);
-    // FIXME: let user specify whether to treat this case as an error or ignore
-    // it as is currently done.
-    if (!EditRange)
-      return SmallVector<Transformation, 0>();
-    auto Replacement = Edit.Replacement(Result);
-    if (!Replacement)
-      return Replacement.takeError();
-    tooling::detail::Transformation T;
-    T.Range = *EditRange;
-    T.Replacement = std::move(*Replacement);
-    Transformations.push_back(std::move(T));
-  }
-  return Transformations;
-}
-
-ASTEdit tooling::change(RangeSelector S, TextGenerator Replacement) {
-  ASTEdit E;
-  E.TargetRange = std::move(S);
-  E.Replacement = std::move(Replacement);
-  return E;
-}
-
-RewriteRule tooling::makeRule(DynTypedMatcher M, SmallVector<ASTEdit, 1> Edits,
-                              TextGenerator Explanation) {
-  return RewriteRule{{RewriteRule::Case{
-      std::move(M), std::move(Edits), std::move(Explanation), {}}}};
-}
-
-void tooling::addInclude(RewriteRule &Rule, StringRef Header,
-                         IncludeFormat Format) {
-  for (auto &Case : Rule.Cases)
-    Case.AddedIncludes.emplace_back(Header.str(), Format);
-}
-
-#ifndef NDEBUG
-// Filters for supported matcher kinds. FIXME: Explicitly list the allowed kinds
-// (all node matcher types except for `QualType` and `Type`), rather than just
-// banning `QualType` and `Type`.
-static bool hasValidKind(const DynTypedMatcher &M) {
-  return !M.canConvertTo<QualType>();
-}
-#endif
-
-// Binds each rule's matcher to a unique (and deterministic) tag based on
-// `TagBase` and the id paired with the case.
-static std::vector<DynTypedMatcher> taggedMatchers(
-    StringRef TagBase,
-    const SmallVectorImpl<std::pair<size_t, RewriteRule::Case>> &Cases) {
-  std::vector<DynTypedMatcher> Matchers;
-  Matchers.reserve(Cases.size());
-  for (const auto &Case : Cases) {
-    std::string Tag = (TagBase + Twine(Case.first)).str();
-    // HACK: Many matchers are not bindable, so ensure that tryBind will work.
-    DynTypedMatcher BoundMatcher(Case.second.Matcher);
-    BoundMatcher.setAllowBind(true);
-    auto M = BoundMatcher.tryBind(Tag);
-    Matchers.push_back(*std::move(M));
-  }
-  return Matchers;
-}
-
-// Simply gathers the contents of the various rules into a single rule. The
-// actual work to combine these into an ordered choice is deferred to matcher
-// registration.
-RewriteRule tooling::applyFirst(ArrayRef<RewriteRule> Rules) {
-  RewriteRule R;
-  for (auto &Rule : Rules)
-    R.Cases.append(Rule.Cases.begin(), Rule.Cases.end());
-  return R;
-}
-
-std::vector<DynTypedMatcher>
-tooling::detail::buildMatchers(const RewriteRule &Rule) {
-  // Map the cases into buckets of matchers -- one for each "root" AST kind,
-  // which guarantees that they can be combined in a single anyOf matcher. Each
-  // case is paired with an identifying number that is converted to a string id
-  // in `taggedMatchers`.
-  std::map<ASTNodeKind, SmallVector<std::pair<size_t, RewriteRule::Case>, 1>>
-      Buckets;
-  const SmallVectorImpl<RewriteRule::Case> &Cases = Rule.Cases;
-  for (int I = 0, N = Cases.size(); I < N; ++I) {
-    assert(hasValidKind(Cases[I].Matcher) &&
-           "Matcher must be non-(Qual)Type node matcher");
-    Buckets[Cases[I].Matcher.getSupportedKind()].emplace_back(I, Cases[I]);
-  }
-
-  std::vector<DynTypedMatcher> Matchers;
-  for (const auto &Bucket : Buckets) {
-    DynTypedMatcher M = DynTypedMatcher::constructVariadic(
-        DynTypedMatcher::VO_AnyOf, Bucket.first,
-        taggedMatchers("Tag", Bucket.second));
-    M.setAllowBind(true);
-    // `tryBind` is guaranteed to succeed, because `AllowBind` was set to true.
-    Matchers.push_back(*M.tryBind(RewriteRule::RootID));
-  }
-  return Matchers;
-}
-
-DynTypedMatcher tooling::detail::buildMatcher(const RewriteRule &Rule) {
-  std::vector<DynTypedMatcher> Ms = buildMatchers(Rule);
-  assert(Ms.size() == 1 && "Cases must have compatible matchers.");
-  return Ms[0];
-}
-
-SourceLocation tooling::detail::getRuleMatchLoc(const MatchResult &Result) {
-  auto &NodesMap = Result.Nodes.getMap();
-  auto Root = NodesMap.find(RewriteRule::RootID);
-  assert(Root != NodesMap.end() && "Transformation failed: missing root node.");
-  llvm::Optional<CharSourceRange> RootRange = getRangeForEdit(
-      CharSourceRange::getTokenRange(Root->second.getSourceRange()),
-      *Result.Context);
-  if (RootRange)
-    return RootRange->getBegin();
-  // The match doesn't have a coherent range, so fall back to the expansion
-  // location as the "beginning" of the match.
-  return Result.SourceManager->getExpansionLoc(
-      Root->second.getSourceRange().getBegin());
-}
-
-// Finds the case that was "selected" -- that is, whose matcher triggered the
-// `MatchResult`.
-const RewriteRule::Case &
-tooling::detail::findSelectedCase(const MatchResult &Result,
-                                  const RewriteRule &Rule) {
-  if (Rule.Cases.size() == 1)
-    return Rule.Cases[0];
-
-  auto &NodesMap = Result.Nodes.getMap();
-  for (size_t i = 0, N = Rule.Cases.size(); i < N; ++i) {
-    std::string Tag = ("Tag" + Twine(i)).str();
-    if (NodesMap.find(Tag) != NodesMap.end())
-      return Rule.Cases[i];
-  }
-  llvm_unreachable("No tag found for this rule.");
-}
-
-constexpr llvm::StringLiteral RewriteRule::RootID;
 
 void Transformer::registerMatchers(MatchFinder *MatchFinder) {
   for (auto &Matcher : tooling::detail::buildMatchers(Rule))
     MatchFinder->addDynamicMatcher(Matcher, this);
 }
 
-void Transformer::run(const MatchResult &Result) {
+void Transformer::run(const MatchFinder::MatchResult &Result) {
   if (Result.Context->getDiagnostics().hasErrorOccurred())
     return;
 
   RewriteRule::Case Case = tooling::detail::findSelectedCase(Result, Rule);
   auto Transformations = tooling::detail::translateEdits(Result, Case.Edits);
   if (!Transformations) {
     Consumer(Transformations.takeError());
     return;
   }
 
   if (Transformations->empty()) {
     // No rewrite applied (but no error encountered either).
-    detail::getRuleMatchLoc(Result).print(
+    tooling::detail::getRuleMatchLoc(Result).print(
         llvm::errs() << "note: skipping match at loc ", *Result.SourceManager);
     llvm::errs() << "\n";
     return;
   }
 
   // Record the results in the AtomicChange, anchored at the location of the
   // first change.
   AtomicChange AC(*Result.SourceManager,