[refactor] add clang-refactor tool with initial testing support and local-rename action

[arphaman]

Summary

This patch depends on https://reviews.llvm.org/D36075 and https://reviews.llvm.org/D36156.

It introduces the clang-refactor tool alongside the local-rename action which is uses the existing renaming engine used by clang-rename. The tool doesn't actually perform the source transformations yet, it just provides testing support. I've moved one test from clang-rename over right now, but will move the others if the general direction of this patch is accepted.

The following options are supported by clang-refactor:

• -v: use verbose output
• -dump: dump results instead of applying them
• -no-dbs: avoid searching/loading databases like compilation database and/or indexer stores/databases
• -selection: The source range that corresponds to the portion of the source that's selected (currently only special command test:<file> is supported).

The testing support provided by clang-refactor is described below:
When -selection=test:<file> is given, clang-refactor will parse the selection commands from that file. The selection commands are grouped and the specified refactoring action invoked by the tool. Each command in a group is expected to produce an identical result. The precise syntax for the selection commands is described in a comment for findTestSelectionRangesIn in TestSupport.h.

Thanks for taking a look!

[arphaman]

Rebase on top of trunk and https://reviews.llvm.org/D37210.

Ping.

[klimek]

One of my main concerns is still that I don't see the need for all the template magic yet :) Why doesn't everybody use the RefactoringResult we define here?

test/Refactor/LocalRename/Field.cpp

4 ↗

(On Diff #113533)

Does this just test the selection?

tools/clang-refactor/ClangRefactor.cpp

135 ↗

(On Diff #113533)

Can we use composition instead of inheritance here?

[arphaman]

In D36574#858763, @klimek wrote:

One of my main concerns is still that I don't see the need for all the template magic yet :) Why doesn't everybody use the RefactoringResult we define here?

This refactoring result is only really useful for the test part of clang-refactor so you can compare two results.

The non-test part of clang-refactor and other clients don't really need an abstract interface for results as they will have different code for different results anyway.

[arphaman]

test/Refactor/LocalRename/Field.cpp

4 ↗

(On Diff #113533)

No, this is the moved clang-rename/Field.cpp test that tests local-rename. I will move the other tests when this patch is accepted.

[arphaman]

Get rid of inheritance of RefactoringEngine by getting rid of the class itself.

[arphaman]

In D36574#858763, @klimek wrote:

One of my main concerns is still that I don't see the need for all the template magic yet :) Why doesn't everybody use the RefactoringResult we define here?

@klimek, are you talking about the template usage in this patch or the whole approach in general? I can probably think of some way to reduce the template boilerplate if you are talking about the general approach.

[klimek]

In D36574#860197, @arphaman wrote:

In D36574#858763, @klimek wrote:

One of my main concerns is still that I don't see the need for all the template magic yet :) Why doesn't everybody use the RefactoringResult we define here?

@klimek, are you talking about the template usage in this patch or the whole approach in general? I can probably think of some way to reduce the template boilerplate if you are talking about the general approach.

Talking about the approach in general :)

test/Refactor/LocalRename/Field.cpp

4 ↗

(On Diff #113533)

Ok, then I find this test really hard to read - where does it check what the symbol was replaced with?

[ioeric]

include/clang/Tooling/Refactoring/RefactoringAction.h

20 ↗

(On Diff #110560)

Please add documentation for the class.

I appreciate your RFC that explains all these concepts; it would be nice if you could move some key concepts into the code documentation since we couldn't expect future readers to refer to the RFC. Thanks! ;)

include/clang/Tooling/Refactoring/SourceSelectionConstraints.h

42 ↗

(On Diff #113736)

when?

55 ↗

(On Diff #113736)

Can we pass RefactoringRuleContext instead of ASTRefactoringContext and get rules can get AST context from the rule context? I'm a bit nervous about having different contexts.

Maybe we should have just one interface T evaluateSelection(RefactoringRuleContext, SelectionConstraint) const. I think the rule context can be useful for all rules.

lib/Tooling/Refactoring/Rename/RenamingAction.cpp

35 ↗

(On Diff #113736)

nit: It's a common practice to wrap implementations in namespaces instead of using using namespace. It becomes hard to manage scopes/namespaces when there are using namespaces.

tools/clang-refactor/ClangRefactor.cpp

60 ↗

(On Diff #110560)

I think you would get a conflict of positional args when you have multiple sub-command instances.

Any reason not to use clang::tooling::CommonOptionsParser which takes care of sources and compilation options for you?

312 ↗

(On Diff #110560)

Do we need a FIXME here? It seems that this simply finds the first command that is available? What if there are more than one commands that satisfy the requirements?

72 ↗

(On Diff #113736)

Same here. There could be potential conflict among subcommands.

Also please provide more detailed description of this option; it is not obvious to users what a selection is.

[arphaman]

In D36574#860203, @klimek wrote:

In D36574#860197, @arphaman wrote:

In D36574#858763, @klimek wrote:

One of my main concerns is still that I don't see the need for all the template magic yet :) Why doesn't everybody use the RefactoringResult we define here?

@klimek, are you talking about the template usage in this patch or the whole approach in general? I can probably think of some way to reduce the template boilerplate if you are talking about the general approach.

Talking about the approach in general :)

I'll work on a patch for a simpler solution as well then. It'll probably be dependent on this one though. I'm pretty sure I need some template magic to figure out how to connect inputs like selection and options and entities that produce source replacements , but I could probably get rid of most of RefactoringActionRule template magic.

include/clang/Tooling/Refactoring/SourceSelectionConstraints.h

42 ↗

(On Diff #113736)

Oops, this isn't supposed to be in this patch, I will remove.

test/Refactor/LocalRename/Field.cpp

4 ↗

(On Diff #113533)

I thought I should check for occurrences, but you're right, it's probably better to always check the source replacements (we can introduce options that apply non-default occurrences in the future which would allow us to check replacements for occurrences in comments).

Would something like //CHECK: "newName" [[@LINE]]:24 -> [[]]:27 work when checking for replacements?

tools/clang-refactor/ClangRefactor.cpp

60 ↗

(On Diff #110560)

Not from my experience, I've tried multiple actions it seems like the right arguments are parsed for the right subcommand. It looks like the cl option parser looks is smart enough to handle identical options across multiple subcommands.

312 ↗

(On Diff #110560)

I don't quite follow, when would multiple subcommands be satisfied? The subcommand corresponds to the action that the user wants to do, there should be no ambiguity.

[klimek]

test/Refactor/LocalRename/Field.cpp

4 ↗

(On Diff #113533)

Yep, I think that'd be better.
Or we could just apply the replacement and check on the replaced code? (I think that's what we do in fixits for clang and clang-tidy)

[arphaman]

• Test by checking the modified source directly. This allowed me to get rid of the refactoring result wrapper as well.
• Remove ASTRefactoringContext
• Address the other review comments

[arphaman]

tools/clang-refactor/ClangRefactor.cpp

60 ↗

(On Diff #110560)

I agree that using CommonOptionsParser would be preferable, but right now it doesn't work well with subcommands. I will create a followup patch that improves subcommand support in CommonOptionsParser and uses them in clang-refactor when this patch is accepted.

[ioeric]

include/clang/Tooling/Refactoring/RefactoringActionRules.h

45 ↗

(On Diff #113804)

Can we get rid of this overload and simply make RefactoringRuleContext mandatory for all refactoring functions?

include/clang/Tooling/Refactoring/RefactoringActionRulesInternal.h

99 ↗

(On Diff #113804)

Do we still need this overload?

include/clang/Tooling/Refactoring/RefactoringRuleContext.h

44 ↗

(On Diff #113804)

Should we also provide an interface to test if this has an ASTContext? Or maybe simply return null if there is ASTContext? WDYT?

lib/Tooling/Refactoring/Rename/RenamingAction.cpp

50 ↗

(On Diff #113804)

Should this be private?

tools/clang-refactor/ClangRefactor.cpp

60 ↗

(On Diff #110560)

Thanks! Could you add a FIXME for the CommonOptionsParser change?

312 ↗

(On Diff #110560)

(I added this comment a while ago, and it seemed to have shifted. I was intended to comment on line 297 auto It = llvm::find_if(. Sorry about that.)

I guess I don't quite understand how a specific subcommand is picked based on the commandline options users provide. The llvm::find_if above seems to simply find the first action/subcommand that is registered? Could you add a comment about how submamands are matched?

[arphaman]

• Always pass in RefactoringRuleContext to the refactoring rule's function.
• Remove now redundant overloads.
• Make LocalRename's actions members private.
• Add CommandLienParser FIXME comment.
• Verify that refactoring actions have unique command names.
• Add a comment that explains how a subcommand is selected.

include/clang/Tooling/Refactoring/RefactoringActionRulesInternal.h

99 ↗

(On Diff #113804)

Removed.

tools/clang-refactor/ClangRefactor.cpp

60 ↗

(On Diff #110560)

Done.

312 ↗

(On Diff #110560)

Ah, I see.

I added a comment in the updated patch that tries to explain the process. Basically we know that one action maps to just one subcommand because the actions must have unique command names. Since we've already created the subcommands, LLVM's command-line parser enables one particular subcommand that was specified in the command-line arguments. This is what this search does, we just look for the enabled subcommand that was turned on by LLVM's command-line parser, without taking any other command-line arguments into account.

[ioeric]

lib/Tooling/Refactoring/Rename/RenamingAction.cpp

43 ↗

(On Diff #113826)

Shouldn't these be public?

tools/clang-refactor/ClangRefactor.cpp

312 ↗

(On Diff #110560)

Ohh! I didn't notice RefactoringActionSubcommand inherits cl::SubCommand. Thanks for the explanation!

79 ↗

(On Diff #113826)

Why return a mutable reference? Also, this is not used in this patch; maybe leave it out?

103 ↗

(On Diff #113826)

Is the test selection temporary before we have true selection?

I am a bit concerned about having test logic in the production code. It makes the code paths a bit complicated, and we might easily end up testing the test logic instead of the actual code logic. I'm fine with this if this is just a short term solution before we have the actual selection support.

tools/clang-refactor/TestSupport.cpp

187 ↗

(On Diff #113826)

I think the ...In suffix is redundant. We could either get rid of it or use ...InFile.

[arphaman]

tools/clang-refactor/ClangRefactor.cpp

103 ↗

(On Diff #113826)

No, both are meant to co-exist. I guess we could introduce a new option (-selection vs -test-selection and hide -test-selection)? Another approach that I was thinking about is to have a special hidden subcommand for each action (e.g. test-local-rename). Alternatively we could use two different tools (clang-refactor vs clang-refactor-test)? Both will have very similar code though.

Note that it's not the first time test logic will exist in the final tool. For example, llvm-cov has a convert-for-testing subcommand that exists in the production tool. I'm not saying that it's necessarily the best option though, but I don't think it's the worst one either ;)

[arphaman]

Address review comments

[ioeric]

tools/clang-refactor/ClangRefactor.cpp

103 ↗

(On Diff #113826)

I guess I am more concerned about mixing production code with testing code than having special options. For example, we have different invocation paths based on ParsedTestSelection. IMO, whether selected ranges come from test selection or actual selection should be transparent to clang-refactor. Maybe refactoring the selection handling logic out as a separate interface would help?

[arphaman]

Refactor the selection argument handling by using a SourceSelectionArgument interface with subclass for test selection and separate out the test code from the tool class.

[arphaman]

tools/clang-refactor/ClangRefactor.cpp

79 ↗

(On Diff #113826)

Removed.

103 ↗

(On Diff #113826)

Makes sense, I just did a cleanup refactoring that should make it better.

[ioeric]

tools/clang-refactor/ClangRefactor.cpp

29 ↗

(On Diff #114125)

Sorry for haven't noticed this earlier, but I think clang::refactor sounds like a better name space than clang::clang_refactor.

62 ↗

(On Diff #114125)

I would expect the SourceSelectionArgument to be more like a container of ranges and further decoupled with the refactoring logic. Same for the TestSelectionRangesInFile.

How about an interface like this?

template <typename T>
void ForAllRanges(T callback) const;

64 ↗

(On Diff #114125)

Do we want to consider other result types?

[arphaman]

• rename namespace
• try to further decouple SourceSelectionArgument

[arphaman]

tools/clang-refactor/ClangRefactor.cpp

62 ↗

(On Diff #114125)

Not sure I fully understand your interface suggestion, but I've tried to decouple some more in the updated patch.

[ioeric]

I think this is ready to go.

@klimek Manuel, are all your concerns addressed?

tools/clang-refactor/ClangRefactor.cpp

62 ↗

(On Diff #114125)

Sorry, I should've been more specific about what I had in mind, but what you have right now is close. I think this can be further decoupled a bit by having forAllRanges take in SourceManeger instead of RefactoringRuleContext since source selection should only care about source ranges.

62 ↗

(On Diff #114309)

IIUC, createCustomConsumer is used to inject testing behavior into clang-refactor. I think this is fine, but please document the intention and the behavior. Thanks!

[ioeric]

Please wait for Manuel's reply before landing the patch ;-)

[arphaman]

@klimek I've added a patch that simplifies the interface and the template code - https://reviews.llvm.org/D37681. Hopefully you'll find that the code there is cleaner.

[ioeric]

Feel free to land the patch now if comments are addressed.

[klimek]

LG. Nice, thanks!

lib/Tooling/Refactoring/Rename/RenamingAction.cpp

42 ↗

(On Diff #114309)

I have to admit, the implementation here is pretty neat! :)

[hokein]

LGTM, a few nits.

lib/Tooling/Refactoring/Rename/RenamingAction.cpp

42 ↗

(On Diff #114309)

+1

test/Refactor/LocalRename/Field.cpp

4 ↗

(On Diff #114309)

I'd use a completed statement in CHECK (like int /*range=*/Bar;), which is more obvious to readers.

test/Refactor/tool-test-support.c

29 ↗

(On Diff #114309)

Maybe add a comment describing the following cases are named test group -- it took me a while to understand why these "invoking action" messages aren't ordered by the original source line number.

tools/clang-refactor/ClangRefactor.cpp

89 ↗

(On Diff #114309)

nit: virtual isn't needed.

tools/clang-refactor/TestSupport.cpp

28 ↗

(On Diff #114309)

I'm not a big fan of using namespace. It would make it hard to find out which namespace is the following method in -- readers have to go to the corresponding header file to find out the namespace. I'm +1 on using a more regular way namespace clang { namespace refactor {...} }.

127 ↗

(On Diff #114309)

nit: virtual can be removed, the same below.

tools/clang-refactor/TestSupport.h

32 ↗

(On Diff #114309)

Do you plan to use refactor on other files?

104 ↗

(On Diff #114309)

s/clang_refactor/refactor

[arphaman]

Oops, I've put in the wrong diff link for r313025. This patch is still not committed.

[arphaman]

tools/clang-refactor/TestSupport.h

32 ↗

(On Diff #114309)

Probably in the future, yeah.

[hokein]

Seems that you have attached a wrong diff, which is committed in r313025.

tools/clang-refactor/TestSupport.h

32 ↗

(On Diff #114309)

SG.

[arphaman]

Yeah, I accidentally committed a prior patch with the link to this diff. I will commit this patch today though so it should be updated.

cfe/trunk/include/clang/Tooling/Refactoring/AtomicChange.h

AtomicChange(llvm::StringRef FilePath, llvm::StringRef Key)
       : Key(Key), FilePath(FilePath) {}
 
   AtomicChange(AtomicChange &&) = default;
   AtomicChange(const AtomicChange &) = default;
 
   AtomicChange &operator=(AtomicChange &&) = default;
   AtomicChange &operator=(const AtomicChange &) = default;
 
+  bool operator==(const AtomicChange &Other) const;
+
   /// \brief Returns the atomic change as a YAML string.
   std::string toYAMLString();
 
   /// \brief Converts a YAML-encoded automic change to AtomicChange.
   static AtomicChange convertFromYAML(llvm::StringRef YAMLContent);
 
   /// \brief Returns the key of this change, which is a concatenation of the
   /// file name and offset of the key position.

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringAction.h

+//===--- RefactoringAction.h - Clang refactoring library ------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_H
+#define LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_H
+
+#include "clang/Basic/LLVM.h"
+#include "clang/Tooling/Refactoring/RefactoringActionRules.h"
+#include <vector>
+
+namespace clang {
+namespace tooling {
+
+/// A refactoring action is a class that defines a set of related refactoring
+/// action rules. These rules get grouped under a common umbrella - a single
+/// clang-refactor subcommand.
+///
+/// A subclass of \c RefactoringAction is responsible for creating the set of
+/// grouped refactoring action rules that represent one refactoring operation.
+/// Although the rules in one action may have a number of different
+/// implementations, they should strive to produce a similar result. It should
+/// be easy for users to identify which refactoring action produced the result
+/// regardless of which refactoring action rule was used.
+///
+/// The distinction between actions and rules enables the creation of action
+/// that uses very different rules, for example:
+///   - local vs global: a refactoring operation like
+///     "add missing switch cases" can be applied to one switch when it's
+///     selected in an editor, or to all switches in a project when an enum
+///     constant is added to an enum.
+///   - tool vs editor: some refactoring operation can be initiated in the
+///     editor when a declaration is selected, or in a tool when the name of
+///     the declaration is passed using a command-line argument.
+class RefactoringAction {
+public:
+  virtual ~RefactoringAction() {}
+
+  /// Returns the name of the subcommand that's used by clang-refactor for this
+  /// action.
+  virtual StringRef getCommand() const = 0;
+
+  virtual StringRef getDescription() const = 0;
+
+  RefactoringActionRules createActiveActionRules();
+
+protected:
+  /// Returns a set of refactoring actions rules that are defined by this
+  /// action.
+  virtual RefactoringActionRules createActionRules() const = 0;
+};
+
+/// Returns the list of all the available refactoring actions.
+std::vector<std::unique_ptr<RefactoringAction>> createRefactoringActions();
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_H

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringActionRegistry.def

+#ifndef REFACTORING_ACTION
+#define REFACTORING_ACTION(Name)
+#endif
+
+REFACTORING_ACTION(LocalRename)
+
+#undef REFACTORING_ACTION

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringActionRule.h

public:
   virtual ~RefactoringActionRule() {}
 
   /// Initiates and performs a specific refactoring action.
   ///
   /// The specific rule will invoke an appropriate \c handle method on a
   /// consumer to propagate the result of the refactoring action.
   virtual void invoke(RefactoringResultConsumer &Consumer,
                       RefactoringRuleContext &Context) = 0;
+
+  /// Returns true when the rule has a source selection requirement that has
+  /// to be fullfilled before refactoring can be performed.
+  virtual bool hasSelectionRequirement() = 0;
 };
 
 /// A set of refactoring action rules that should have unique initiation
 /// requirements.
 using RefactoringActionRules =
     std::vector<std::unique_ptr<RefactoringActionRule>>;
 
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_RULE_H

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringActionRuleRequirementsInternal.h

struct SourceSelectionRequirement
   /// an OutputT if the selection requirement was successfully. The OutpuT
   /// value is wrapped in Expected<Optional<>> which is then unwrapped by the
   /// refactoring action rule before passing the value to the refactoring
   /// function.
   Expected<Optional<OutputType>> evaluate(RefactoringRuleContext &Context) {
     Optional<InputT> Value = InputT::evaluate(Context);
     if (!Value)
       return None;
-    return std::move(Requirement.evaluateSelection(*Value));
+    return std::move(Requirement.evaluateSelection(Context, *Value));
   }
 
 private:
   const RequirementT Requirement;
 };
 
 } // end namespace internal
 
 namespace traits {
 
 /// A type trait that returns true iff the given type is a valid rule
 /// requirement.
 template <typename First, typename... Rest>
 struct IsRequirement : std::conditional<IsRequirement<First>::value &&
                                             IsRequirement<Rest...>::value,
                                         std::true_type, std::false_type>::type {
 };
 
 template <typename T>
 struct IsRequirement<T>
     : std::conditional<std::is_base_of<internal::RequirementBase, T>::value,
                        std::true_type, std::false_type>::type {};
 
+/// A type trait that returns true when the given type has at least one source
+/// selection requirement.
+template <typename First, typename... Rest>
+struct HasSelectionRequirement
+    : std::conditional<HasSelectionRequirement<First>::value ||
+                           HasSelectionRequirement<Rest...>::value,
+                       std::true_type, std::false_type>::type {};
+
+template <typename I, typename O, typename R>
+struct HasSelectionRequirement<internal::SourceSelectionRequirement<I, O, R>>
+    : std::true_type {};
+
+template <typename T> struct HasSelectionRequirement<T> : std::false_type {};
+
 } // end namespace traits
 } // end namespace refactoring_action_rules
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_REQUIREMENTS_INTERNAL_H

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringActionRules.h

 #ifndef LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_RULES_H
 #define LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_RULES_H
 
 #include "clang/Tooling/Refactoring/RefactoringActionRule.h"
 #include "clang/Tooling/Refactoring/RefactoringActionRulesInternal.h"
 
 namespace clang {
 namespace tooling {
+
+class RefactoringRuleContext;
+
 namespace refactoring_action_rules {
 
 /// Creates a new refactoring action rule that invokes the given function once
 /// all of the requirements are satisfied. The values produced during the
 /// evaluation of requirements are passed to the given function (in the order of
 /// requirements).
 ///
 /// \param RefactoringFunction the function that will perform the refactoring

 /// \c traits::IsRequirement<T> must be true. The following requirements are
 /// currently supported:
 ///
 ///  - requiredSelection: The refactoring function won't be invoked unless the
 ///                       given selection requirement is satisfied.
 template <typename ResultType, typename... RequirementTypes>
 std::unique_ptr<RefactoringActionRule>
 createRefactoringRule(Expected<ResultType> (*RefactoringFunction)(
+                          const RefactoringRuleContext &,
                           typename RequirementTypes::OutputType...),
                       const RequirementTypes &... Requirements) {
   static_assert(tooling::traits::IsValidRefactoringResult<ResultType>::value,
                 "invalid refactoring result type");
   static_assert(traits::IsRequirement<RequirementTypes...>::value,
                 "invalid refactoring action rule requirement");
   return llvm::make_unique<internal::PlainFunctionRule<
       decltype(RefactoringFunction), RequirementTypes...>>(
       RefactoringFunction, std::make_tuple(Requirements...));
 }
 
 template <
     typename Callable, typename... RequirementTypes,
     typename Fn = decltype(&Callable::operator()),
     typename ResultType = typename internal::LambdaDeducer<Fn>::ReturnType,
     bool IsNonCapturingLambda = std::is_convertible<
-        Callable,
-        ResultType (*)(typename RequirementTypes::OutputType...)>::value,
+        Callable, typename internal::LambdaDeducer<Fn>::FunctionType>::value,
     typename = typename std::enable_if<IsNonCapturingLambda>::type>
 std::unique_ptr<RefactoringActionRule>
 createRefactoringRule(const Callable &C,
                       const RequirementTypes &... Requirements) {
-  ResultType (*Func)(typename RequirementTypes::OutputType...) = C;
+  typename internal::LambdaDeducer<Fn>::FunctionType Func = C;
   return createRefactoringRule(Func, Requirements...);
 }
 
 } // end namespace refactoring_action_rules
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_RULES_H

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringActionRulesInternal.h

PlainFunctionRule(FunctionType Function,
       : Function(Function), Requirements(std::move(Requirements)) {}
 
   void invoke(RefactoringResultConsumer &Consumer,
               RefactoringRuleContext &Context) override {
     return invokeImpl(Consumer, Context,
                       llvm::index_sequence_for<RequirementTypes...>());
   }
 
+  bool hasSelectionRequirement() override {
+    return traits::HasSelectionRequirement<RequirementTypes...>::value;
+  }
+
 private:
   /// Returns \c T when given \c Expected<Optional<T>>, or \c T otherwise.
   template <typename T>
   static T &&unwrapRequirementResult(Expected<Optional<T>> &&X) {
     assert(X && "unexpected diagnostic!");
     return std::move(**X);
   }
   template <typename T> static T &&unwrapRequirementResult(T &&X) {
static Optional<llvm::Error> takeErrorNone(Expected<Optional<T>> &Diag) {
     if (!*Diag)
       return llvm::Error::success(); // Initiation failed without a diagnostic.
     return None;
   }
 
   template <size_t... Is>
   void invokeImpl(RefactoringResultConsumer &Consumer,
                   RefactoringRuleContext &Context,
-                  llvm::index_sequence<Is...>) {
+                  llvm::index_sequence<Is...> Seq) {
     // Initiate the operation.
     auto Values =
         std::make_tuple(std::get<Is>(Requirements).evaluate(Context)...);
     Optional<llvm::Error> InitiationFailure =
         findErrorNone(std::get<Is>(Values)...);
     if (InitiationFailure) {
       llvm::Error Error = std::move(*InitiationFailure);
       if (!Error)
         // FIXME: Use a diagnostic.
         return Consumer.handleError(llvm::make_error<llvm::StringError>(
             "refactoring action can't be initiated with the specified "
             "selection range",
             llvm::inconvertibleErrorCode()));
       return Consumer.handleError(std::move(Error));
     }
     // Perform the operation.
-    auto Result =
-        Function(unwrapRequirementResult(std::move(std::get<Is>(Values)))...);
+    auto Result = Function(
+        Context, unwrapRequirementResult(std::move(std::get<Is>(Values)))...);
     if (!Result)
       return Consumer.handleError(Result.takeError());
     Consumer.handle(std::move(*Result));
   }
 
   FunctionType Function;
   std::tuple<RequirementTypes...> Requirements;
 };
 
 /// Used to deduce the refactoring result type for the lambda that passed into
 /// createRefactoringRule.
 template <typename T> struct LambdaDeducer;
 template <typename T, typename R, typename... Args>
-struct LambdaDeducer<R (T::*)(Args...) const> {
+struct LambdaDeducer<R (T::*)(const RefactoringRuleContext &, Args...) const> {
   using ReturnType = R;
+  using FunctionType = R (*)(const RefactoringRuleContext &, Args...);
 };
 
 } // end namespace internal
 } // end namespace refactoring_action_rules
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_ACTION_RULES_INTERNAL_H

cfe/trunk/include/clang/Tooling/Refactoring/RefactoringRuleContext.h

 //===--- RefactoringRuleContext.h - Clang refactoring library -------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_RULE_CONTEXT_H
 #define LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_RULE_CONTEXT_H
 
 #include "clang/Basic/SourceManager.h"
 
 namespace clang {
+
+class ASTContext;
+
 namespace tooling {
 
 /// The refactoring rule context stores all of the inputs that might be needed
 /// by a refactoring action rule. It can create the specialized
 /// \c ASTRefactoringOperation or \c PreprocessorRefactoringOperation values
 /// that can be used by the refactoring action rules.
 ///
 /// The following inputs are stored by the operation:
public:
   const SourceManager &getSources() const { return SM; }
 
   /// Returns the current source selection range as set by the
   /// refactoring engine. Can be invalid.
   SourceRange getSelectionRange() const { return SelectionRange; }
 
   void setSelectionRange(SourceRange R) { SelectionRange = R; }
 
+  bool hasASTContext() const { return AST; }
+
+  ASTContext &getASTContext() const {
+    assert(AST && "no AST!");
+    return *AST;
+  }
+
+  void setASTContext(ASTContext &Context) { AST = &Context; }
+
 private:
   /// The source manager for the translation unit / file on which a refactoring
   /// action might operate on.
   const SourceManager &SM;
   /// An optional source selection range that's commonly used to represent
   /// a selection in an editor.
   SourceRange SelectionRange;
+  /// An optional AST for the translation unit on which a refactoring action
+  /// might operate on.
+  ASTContext *AST = nullptr;
 };
 
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_REFACTOR_REFACTORING_RULE_CONTEXT_H

cfe/trunk/include/clang/Tooling/Refactoring/Rename/USRFindingAction.h

 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/ArrayRef.h"
 
 #include <string>
 #include <vector>
 
 namespace clang {
 class ASTConsumer;
+class ASTContext;
 class CompilerInstance;
 class NamedDecl;
 
 namespace tooling {
 
 /// Returns the canonical declaration that best represents a symbol that can be
 /// renamed.
 ///
 /// The following canonicalization rules are currently used:
 ///
 /// - A constructor is canonicalized to its class.
 /// - A destructor is canonicalized to its class.
 const NamedDecl *getCanonicalSymbolDeclaration(const NamedDecl *FoundDecl);
 
+/// Returns the set of USRs that correspond to the given declaration.
+std::vector<std::string> getUSRsForDeclaration(const NamedDecl *ND,
+                                               ASTContext &Context);
+
 struct USRFindingAction {
   USRFindingAction(ArrayRef<unsigned> SymbolOffsets,
                    ArrayRef<std::string> QualifiedNames, bool Force)
       : SymbolOffsets(SymbolOffsets), QualifiedNames(QualifiedNames),
         ErrorOccurred(false), Force(Force) {}
   std::unique_ptr<ASTConsumer> newASTConsumer();
 
   ArrayRef<std::string> getUSRSpellings() { return SpellingNames; }

cfe/trunk/include/clang/Tooling/Refactoring/SourceSelectionConstraints.h

 //===--- SourceSelectionConstraints.h - Clang refactoring library ---------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLING_REFACTOR_SOURCE_SELECTION_CONSTRAINTS_H
 #define LLVM_CLANG_TOOLING_REFACTOR_SOURCE_SELECTION_CONSTRAINTS_H
 
 #include "clang/Basic/LLVM.h"
 #include "clang/Basic/SourceLocation.h"
+#include "clang/Tooling/Refactoring/RefactoringRuleContext.h"
 #include <type_traits>
 
 namespace clang {
 namespace tooling {
 
 class RefactoringRuleContext;
 
 namespace selection {

 
 private:
   const SourceManager &SM;
   SourceRange Range;
 };
 
 /// A custom selection requirement.
 class Requirement {
-  /// Subclasses must implement 'T evaluateSelection(SelectionConstraint) const'
-  /// member function. \c T is used to determine the return type that is
-  /// passed to the refactoring rule's function.
+  /// Subclasses must implement
+  /// 'T evaluateSelection(const RefactoringRuleContext &,
+  /// SelectionConstraint) const' member function. \c T is used to determine
+  /// the return type that is passed to the refactoring rule's function.
   /// If T is \c DiagnosticOr<S> , then \c S is passed to the rule's function
   /// using move semantics.
   /// Otherwise, T is passed to the function directly using move semantics.
   ///
   /// The different return type rules allow refactoring actions to fail
   /// initiation when the relevant portions of AST aren't selected.
 };
 
 namespace traits {
 
 /// A type trait that returns true iff the given type is a valid selection
 /// constraint.
 template <typename T> struct IsConstraint : public std::false_type {};
 
 } // end namespace traits
 
 namespace internal {
 
 template <typename T> struct EvaluateSelectionChecker : std::false_type {};
 
 template <typename T, typename R, typename A>
-struct EvaluateSelectionChecker<R (T::*)(A) const> : std::true_type {
+struct EvaluateSelectionChecker<R (T::*)(const RefactoringRuleContext &, A)
+                                    const> : std::true_type {
   using ReturnType = R;
   using ArgType = A;
 };
 
 template <typename T> class Identity : public Requirement {
 public:
-  T evaluateSelection(T Value) const { return std::move(Value); }
+  T evaluateSelection(const RefactoringRuleContext &, T Value) const {
+    return std::move(Value);
+  }
 };
 
 } // end namespace internal
 
 /// A identity function that returns the given selection constraint is provided
 /// for convenience, as it can be passed to \c requiredSelection directly.
 template <typename T> internal::Identity<T> identity() {
   static_assert(

cfe/trunk/include/clang/module.modulemap

 
 module Clang_Tooling {
   requires cplusplus umbrella "Tooling" module * { export * }
   // FIXME: Exclude these headers to avoid pulling all of the AST matchers
   // library into clang-format. Due to inline key functions in the headers,
   // importing the AST matchers library gives a link dependency on the AST
   // matchers (and thus the AST), which clang-format should not have.
   exclude header "Tooling/RefactoringCallbacks.h"
+
+  textual header "Tooling/Refactoring/RefactoringActionRegistry.def"
 }
 
 module Clang_ToolingCore {
   requires cplusplus
   umbrella "Tooling/Core" module * { export * }
 }

cfe/trunk/lib/Tooling/Refactoring/AtomicChange.cpp

AtomicChange::AtomicChange(std::string Key, std::string FilePath,
                            std::vector<std::string> InsertedHeaders,
                            std::vector<std::string> RemovedHeaders,
                            clang::tooling::Replacements Replaces)
     : Key(std::move(Key)), FilePath(std::move(FilePath)),
       Error(std::move(Error)), InsertedHeaders(std::move(InsertedHeaders)),
       RemovedHeaders(std::move(RemovedHeaders)), Replaces(std::move(Replaces)) {
 }
 
+bool AtomicChange::operator==(const AtomicChange &Other) const {
+  if (Key != Other.Key || FilePath != Other.FilePath || Error != Other.Error)
+    return false;
+  if (!(Replaces == Other.Replaces))
+    return false;
+  // FXIME: Compare header insertions/removals.
+  return true;
+}
+
 std::string AtomicChange::toYAMLString() {
   std::string YamlContent;
   llvm::raw_string_ostream YamlContentStream(YamlContent);
 
   llvm::yaml::Output YAML(YamlContentStream);
   YAML << *this;
   YamlContentStream.flush();
   return YamlContent;

cfe/trunk/lib/Tooling/Refactoring/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS Support)
 
 add_clang_library(clangToolingRefactor
   ASTSelection.cpp
   AtomicChange.cpp
+  RefactoringActions.cpp
   Rename/RenamingAction.cpp
   Rename/SymbolOccurrences.cpp
   Rename/USRFinder.cpp
   Rename/USRFindingAction.cpp
   Rename/USRLocFinder.cpp
   SourceSelectionConstraints.cpp
 
   LINK_LIBS
   clangAST
   clangASTMatchers
   clangBasic
   clangFormat
   clangIndex
   clangLex
   clangToolingCore
   )

cfe/trunk/lib/Tooling/Refactoring/RefactoringActions.cpp

+//===--- RefactoringActions.cpp - Constructs refactoring actions ----------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/Refactoring/RefactoringAction.h"
+
+namespace clang {
+namespace tooling {
+
+// Forward declare the individual create*Action functions.
+#define REFACTORING_ACTION(Name)                                               \
+  std::unique_ptr<RefactoringAction> create##Name##Action();
+#include "clang/Tooling/Refactoring/RefactoringActionRegistry.def"
+
+std::vector<std::unique_ptr<RefactoringAction>> createRefactoringActions() {
+  std::vector<std::unique_ptr<RefactoringAction>> Actions;
+
+#define REFACTORING_ACTION(Name) Actions.push_back(create##Name##Action());
+#include "clang/Tooling/Refactoring/RefactoringActionRegistry.def"
+
+  return Actions;
+}
+
+RefactoringActionRules RefactoringAction::createActiveActionRules() {
+  // FIXME: Filter out rules that are not supported by a particular client.
+  return createActionRules();
+}
+
+} // end namespace tooling
+} // end namespace clang

cfe/trunk/lib/Tooling/Refactoring/Rename/RenamingAction.cpp

 #include "clang/AST/ASTContext.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendAction.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Tooling/CommonOptionsParser.h"
 #include "clang/Tooling/Refactoring.h"
+#include "clang/Tooling/Refactoring/RefactoringAction.h"
+#include "clang/Tooling/Refactoring/RefactoringActionRules.h"
+#include "clang/Tooling/Refactoring/Rename/USRFinder.h"
+#include "clang/Tooling/Refactoring/Rename/USRFindingAction.h"
 #include "clang/Tooling/Refactoring/Rename/USRLocFinder.h"
 #include "clang/Tooling/Tooling.h"
 #include "llvm/ADT/STLExtras.h"
 #include <string>
 #include <vector>
 
 using namespace llvm;
 
 namespace clang {
 namespace tooling {
 
+namespace {
+
+class LocalRename : public RefactoringAction {
+public:
+  StringRef getCommand() const override { return "local-rename"; }
+
+  StringRef getDescription() const override {
+    return "Finds and renames symbols in code with no indexer support";
+  }
+
+  /// Returns a set of refactoring actions rules that are defined by this
+  /// action.
+  RefactoringActionRules createActionRules() const override {
+    using namespace refactoring_action_rules;
+    RefactoringActionRules Rules;
+    Rules.push_back(createRefactoringRule(
+        renameOccurrences, requiredSelection(SymbolSelectionRequirement())));
+    return Rules;
+  }
+
+private:
+  static Expected<AtomicChanges>
+  renameOccurrences(const RefactoringRuleContext &Context,
+                    const NamedDecl *ND) {
+    std::vector<std::string> USRs =
+        getUSRsForDeclaration(ND, Context.getASTContext());
+    std::string PrevName = ND->getNameAsString();
+    auto Occurrences = getOccurrencesOfUSRs(
+        USRs, PrevName, Context.getASTContext().getTranslationUnitDecl());
+
+    // FIXME: This is a temporary workaround that's needed until the refactoring
+    // options are implemented.
+    StringRef NewName = "Bar";
+    return createRenameReplacements(
+        Occurrences, Context.getASTContext().getSourceManager(), NewName);
+  }
+
+  class SymbolSelectionRequirement : public selection::Requirement {
+  public:
+    Expected<Optional<const NamedDecl *>>
+    evaluateSelection(const RefactoringRuleContext &Context,
+                      selection::SourceSelectionRange Selection) const {
+      const NamedDecl *ND = getNamedDeclAt(Context.getASTContext(),
+                                           Selection.getRange().getBegin());
+      if (!ND)
+        return None;
+      return getCanonicalSymbolDeclaration(ND);
+    }
+  };
+};
+
+} // end anonymous namespace
+
+std::unique_ptr<RefactoringAction> createLocalRenameAction() {
+  return llvm::make_unique<LocalRename>();
+}
+
 Expected<std::vector<AtomicChange>>
 createRenameReplacements(const SymbolOccurrences &Occurrences,
                          const SourceManager &SM,
                          ArrayRef<StringRef> NewNameStrings) {
   // FIXME: A true local rename can use just one AtomicChange.
   std::vector<AtomicChange> Changes;
   for (const auto &Occurrence : Occurrences) {
     ArrayRef<SourceRange> Ranges = Occurrence.getNameRanges();

cfe/trunk/lib/Tooling/Refactoring/Rename/USRFindingAction.cpp

private:
   const Decl *FoundDecl;
   ASTContext &Context;
   std::set<std::string> USRSet;
   std::vector<const CXXMethodDecl *> OverriddenMethods;
   std::vector<const ClassTemplatePartialSpecializationDecl *> PartialSpecs;
 };
 } // namespace
 
+std::vector<std::string> getUSRsForDeclaration(const NamedDecl *ND,
+                                               ASTContext &Context) {
+  AdditionalUSRFinder Finder(ND, Context);
+  return Finder.Find();
+}
+
 class NamedDeclFindingConsumer : public ASTConsumer {
 public:
   NamedDeclFindingConsumer(ArrayRef<unsigned> SymbolOffsets,
                            ArrayRef<std::string> QualifiedNames,
                            std::vector<std::string> &SpellingNames,
                            std::vector<std::vector<std::string>> &USRList,
                            bool Force, bool &ErrorOccurred)
       : SymbolOffsets(SymbolOffsets), QualifiedNames(QualifiedNames),

cfe/trunk/test/CMakeLists.txt

 list(APPEND CLANG_TEST_DEPS
   clang clang-headers
   clang-format
   c-index-test diagtool
   clang-tblgen
   clang-offload-bundler
   clang-import-test
   clang-rename
+  clang-refactor
   clang-diff
   )
   
 if(CLANG_ENABLE_STATIC_ANALYZER)
   list(APPEND CLANG_TEST_DEPS
     clang-check
     )
 endif()

cfe/trunk/test/Refactor/LocalRename/Field.cpp

+// RUN: clang-refactor local-rename -selection=test:%s -no-dbs %s | FileCheck %s
+
+class Baz {
+  int /*range=*/Foo; // CHECK: int /*range=*/Bar;
+public:
+  Baz();
+};
+
+Baz::Baz() : /*range=*/Foo(0) {}  // CHECK: Baz::Baz() : /*range=*/Bar(0) {};

cfe/trunk/test/Refactor/tool-common-options.c

+// RUN: not clang-refactor 2>&1 | FileCheck --check-prefix=MISSING_ACTION %s
+// MISSING_ACTION: error: no refactoring action given
+// MISSING_ACTION-NEXT: note: the following actions are supported:
+
+// RUN: not clang-refactor local-rename -no-dbs 2>&1 | FileCheck --check-prefix=MISSING_SOURCES %s
+// MISSING_SOURCES: error: must provide paths to the source files when '-no-dbs' is used

cfe/trunk/test/Refactor/tool-test-support.c

+// RUN: clang-refactor local-rename -selection=test:%s -no-dbs -v %s 2>&1 | FileCheck %s
+
+/*range=*/int test;
+
+/*range named=*/int test2;
+
+/*range= +1*/int test3;
+
+/* range = +100 */int test4;
+
+/*range named =+0*/int test5;
+
+// CHECK: Test selection group '':
+// CHECK-NEXT:   100-100
+// CHECK-NEXT:   153-153
+// CHECK-NEXT:   192-192
+// CHECK-NEXT: Test selection group 'named':
+// CHECK-NEXT:   127-127
+// CHECK-NEXT:   213-213
+
+// The following invocations are in the default group:
+
+// CHECK: invoking action 'local-rename':
+// CHECK-NEXT: -selection={{.*}}tool-test-support.c:3:11
+
+// CHECK: invoking action 'local-rename':
+// CHECK-NEXT: -selection={{.*}}tool-test-support.c:7:15
+
+// CHECK: invoking action 'local-rename':
+// CHECK-NEXT: -selection={{.*}}tool-test-support.c:9:29
+
+
+// The following invocations are in the 'named' group, and they follow
+// the default invocation even if some of their ranges occur prior to the
+// ranges from the default group because the groups are tested one-by-one:
+
+// CHECK: invoking action 'local-rename':
+// CHECK-NEXT: -selection={{.*}}tool-test-support.c:5:17
+
+// CHECK: invoking action 'local-rename':
+// CHECK-NEXT: -selection={{.*}}tool-test-support.c:11:20

cfe/trunk/test/clang-rename/Field.cpp

-class Baz {
-  int Foo; /* Test 1 */ // CHECK: int Bar;
-public:
-  Baz();
-};
-
-Baz::Baz() : Foo(0) /* Test 2 */ {}  // CHECK: Baz::Baz() : Bar(0)
-
-// Test 1.
-// RUN: clang-rename -offset=18 -new-name=Bar %s -- | sed 's,//.*,,' | FileCheck %s
-// Test 2.
-// RUN: clang-rename -offset=89 -new-name=Bar %s -- | sed 's,//.*,,' | FileCheck %s
-
-// To find offsets after modifying the file, use:
-//   grep -Ubo 'Foo.*' <file>

cfe/trunk/tools/CMakeLists.txt

 create_subdirectory_options(CLANG TOOL)
 
 add_clang_subdirectory(diagtool)
 add_clang_subdirectory(driver)
 add_clang_subdirectory(clang-diff)
 add_clang_subdirectory(clang-format)
 add_clang_subdirectory(clang-format-vs)
 add_clang_subdirectory(clang-fuzzer)
 add_clang_subdirectory(clang-import-test)
 add_clang_subdirectory(clang-offload-bundler)
 
 add_clang_subdirectory(c-index-test)
 
 add_clang_subdirectory(clang-rename)
+add_clang_subdirectory(clang-refactor)
 
 if(CLANG_ENABLE_ARCMT)
   add_clang_subdirectory(arcmt-test)
   add_clang_subdirectory(c-arcmt-test)
 endif()
 
 if(CLANG_ENABLE_STATIC_ANALYZER)
   add_clang_subdirectory(clang-check)

cfe/trunk/tools/clang-refactor/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Option
+  Support
+  )
+
+add_clang_executable(clang-refactor
+  ClangRefactor.cpp
+  TestSupport.cpp
+  )
+
+target_link_libraries(clang-refactor
+  clangBasic
+  clangFrontend
+  clangRewrite
+  clangTooling
+  clangToolingCore
+  clangToolingRefactor
+  )
+
+install(TARGETS clang-refactor RUNTIME DESTINATION bin)

cfe/trunk/tools/clang-refactor/ClangRefactor.cpp

+//===--- ClangRefactor.cpp - Clang-based refactoring tool -----------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief This file implements a clang-refactor tool that performs various
+/// source transformations.
+///
+//===----------------------------------------------------------------------===//
+
+#include "TestSupport.h"
+#include "clang/Rewrite/Core/Rewriter.h"
+#include "clang/Tooling/Refactoring.h"
+#include "clang/Tooling/Refactoring/RefactoringAction.h"
+#include "clang/Tooling/Refactoring/Rename/RenamingAction.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/raw_ostream.h"
+#include <string>
+
+using namespace clang;
+using namespace tooling;
+using namespace refactor;
+namespace cl = llvm::cl;
+
+namespace opts {
+
+static cl::OptionCategory CommonRefactorOptions("Common refactoring options");
+
+static cl::opt<bool>
+    NoDatabases("no-dbs",
+                cl::desc("Ignore external databases including Clang's "
+                         "compilation database and indexer stores"),
+                cl::cat(CommonRefactorOptions), cl::sub(*cl::AllSubCommands));
+
+static cl::opt<bool> Verbose("v", cl::desc("Use verbose output"),
+                             cl::cat(CommonRefactorOptions),
+                             cl::sub(*cl::AllSubCommands));
+} // end namespace opts
+
+namespace {
+
+/// Stores the parsed `-selection` argument.
+class SourceSelectionArgument {
+public:
+  virtual ~SourceSelectionArgument() {}
+
+  /// Parse the `-selection` argument.
+  ///
+  /// \returns A valid argument when the parse succedeed, null otherwise.
+  static std::unique_ptr<SourceSelectionArgument> fromString(StringRef Value);
+
+  /// Prints any additional state associated with the selection argument to
+  /// the given output stream.
+  virtual void print(raw_ostream &OS) = 0;
+
+  /// Returns a replacement refactoring result consumer (if any) that should
+  /// consume the results of a refactoring operation.
+  ///
+  /// The replacement refactoring result consumer is used by \c
+  /// TestSourceSelectionArgument to inject a test-specific result handling
+  /// logic into the refactoring operation. The test-specific consumer
+  /// ensures that the individual results in a particular test group are
+  /// identical.
+  virtual std::unique_ptr<RefactoringResultConsumer> createCustomConsumer() {
+    return nullptr;
+  }
+
+  /// Runs the give refactoring function for each specified selection.
+  ///
+  /// \returns true if an error occurred, false otherwise.
+  virtual bool
+  forAllRanges(const SourceManager &SM,
+               llvm::function_ref<void(SourceRange R)> Callback) = 0;
+};
+
+/// Stores the parsed -selection=test:<filename> option.
+class TestSourceSelectionArgument final : public SourceSelectionArgument {
+public:
+  TestSourceSelectionArgument(TestSelectionRangesInFile TestSelections)
+      : TestSelections(std::move(TestSelections)) {}
+
+  void print(raw_ostream &OS) override { TestSelections.dump(OS); }
+
+  std::unique_ptr<RefactoringResultConsumer> createCustomConsumer() override {
+    return TestSelections.createConsumer();
+  }
+
+  /// Testing support: invokes the selection action for each selection range in
+  /// the test file.
+  bool forAllRanges(const SourceManager &SM,
+                    llvm::function_ref<void(SourceRange R)> Callback) override {
+    return TestSelections.foreachRange(SM, Callback);
+  }
+
+private:
+  TestSelectionRangesInFile TestSelections;
+};
+
+std::unique_ptr<SourceSelectionArgument>
+SourceSelectionArgument::fromString(StringRef Value) {
+  if (Value.startswith("test:")) {
+    StringRef Filename = Value.drop_front(strlen("test:"));
+    Optional<TestSelectionRangesInFile> ParsedTestSelection =
+        findTestSelectionRanges(Filename);
+    if (!ParsedTestSelection)
+      return nullptr; // A parsing error was already reported.
+    return llvm::make_unique<TestSourceSelectionArgument>(
+        std::move(*ParsedTestSelection));
+  }
+  // FIXME: Support true selection ranges.
+  llvm::errs() << "error: '-selection' option must be specified using "
+                  "<file>:<line>:<column> or "
+                  "<file>:<line>:<column>-<line>:<column> format";
+  return nullptr;
+}
+
+/// A subcommand that corresponds to individual refactoring action.
+class RefactoringActionSubcommand : public cl::SubCommand {
+public:
+  RefactoringActionSubcommand(std::unique_ptr<RefactoringAction> Action,
+                              RefactoringActionRules ActionRules,
+                              cl::OptionCategory &Category)
+      : SubCommand(Action->getCommand(), Action->getDescription()),
+        Action(std::move(Action)), ActionRules(std::move(ActionRules)) {
+    Sources = llvm::make_unique<cl::list<std::string>>(
+        cl::Positional, cl::ZeroOrMore, cl::desc("<source0> [... <sourceN>]"),
+        cl::cat(Category), cl::sub(*this));
+
+    // Check if the selection option is supported.
+    bool HasSelection = false;
+    for (const auto &Rule : this->ActionRules) {
+      if ((HasSelection = Rule->hasSelectionRequirement()))
+        break;
+    }
+    if (HasSelection) {
+      Selection = llvm::make_unique<cl::opt<std::string>>(
+          "selection",
+          cl::desc("The selected source range in which the refactoring should "
+                   "be initiated (<file>:<line>:<column>-<line>:<column> or "
+                   "<file>:<line>:<column>)"),
+          cl::cat(Category), cl::sub(*this));
+    }
+  }
+
+  ~RefactoringActionSubcommand() { unregisterSubCommand(); }
+
+  const RefactoringActionRules &getActionRules() const { return ActionRules; }
+
+  /// Parses the command-line arguments that are specific to this rule.
+  ///
+  /// \returns true on error, false otherwise.
+  bool parseArguments() {
+    if (Selection) {
+      ParsedSelection = SourceSelectionArgument::fromString(*Selection);
+      if (!ParsedSelection)
+        return true;
+    }
+    return false;
+  }
+
+  SourceSelectionArgument *getSelection() const {
+    assert(Selection && "selection not supported!");
+    return ParsedSelection.get();
+  }
+
+  ArrayRef<std::string> getSources() const { return *Sources; }
+
+private:
+  std::unique_ptr<RefactoringAction> Action;
+  RefactoringActionRules ActionRules;
+  std::unique_ptr<cl::list<std::string>> Sources;
+  std::unique_ptr<cl::opt<std::string>> Selection;
+  std::unique_ptr<SourceSelectionArgument> ParsedSelection;
+};
+
+class ClangRefactorConsumer : public RefactoringResultConsumer {
+public:
+  void handleError(llvm::Error Err) {
+    llvm::errs() << llvm::toString(std::move(Err)) << "\n";
+  }
+
+  // FIXME: Consume atomic changes and apply them to files.
+};
+
+class ClangRefactorTool {
+public:
+  std::vector<std::unique_ptr<RefactoringActionSubcommand>> SubCommands;
+
+  ClangRefactorTool() {
+    std::vector<std::unique_ptr<RefactoringAction>> Actions =
+        createRefactoringActions();
+
+    // Actions must have unique command names so that we can map them to one
+    // subcommand.
+    llvm::StringSet<> CommandNames;
+    for (const auto &Action : Actions) {
+      if (!CommandNames.insert(Action->getCommand()).second) {
+        llvm::errs() << "duplicate refactoring action command '"
+                     << Action->getCommand() << "'!";
+        exit(1);
+      }
+    }
+
+    // Create subcommands and command-line options.
+    for (auto &Action : Actions) {
+      SubCommands.push_back(llvm::make_unique<RefactoringActionSubcommand>(
+          std::move(Action), Action->createActiveActionRules(),
+          opts::CommonRefactorOptions));
+    }
+  }
+
+  using TUCallbackType = llvm::function_ref<void(ASTContext &)>;
+
+  /// Parses the translation units that were given to the subcommand using
+  /// the 'sources' option and invokes the callback for each parsed
+  /// translation unit.
+  bool foreachTranslationUnit(RefactoringActionSubcommand &Subcommand,
+                              TUCallbackType Callback) {
+    std::unique_ptr<CompilationDatabase> Compilations;
+    if (opts::NoDatabases) {
+      // FIXME (Alex L): Support compilation options.
+      Compilations =
+          llvm::make_unique<clang::tooling::FixedCompilationDatabase>(
+              ".", std::vector<std::string>());
+    } else {
+      // FIXME (Alex L): Support compilation database.
+      llvm::errs() << "compilation databases are not supported yet!\n";
+      return true;
+    }
+
+    class ToolASTConsumer : public ASTConsumer {
+    public:
+      TUCallbackType Callback;
+      ToolASTConsumer(TUCallbackType Callback) : Callback(Callback) {}
+
+      void HandleTranslationUnit(ASTContext &Context) override {
+        Callback(Context);
+      }
+    };
+    class ActionWrapper {
+    public:
+      TUCallbackType Callback;
+      ActionWrapper(TUCallbackType Callback) : Callback(Callback) {}
+
+      std::unique_ptr<ASTConsumer> newASTConsumer() {
+        return llvm::make_unique<ToolASTConsumer>(std::move(Callback));
+      }
+    };
+
+    ClangTool Tool(*Compilations, Subcommand.getSources());
+    ActionWrapper ToolAction(std::move(Callback));
+    std::unique_ptr<tooling::FrontendActionFactory> Factory =
+        tooling::newFrontendActionFactory(&ToolAction);
+    return Tool.run(Factory.get());
+  }
+
+  /// Logs an individual refactoring action invocation to STDOUT.
+  void logInvocation(RefactoringActionSubcommand &Subcommand,
+                     const RefactoringRuleContext &Context) {
+    if (!opts::Verbose)
+      return;
+    llvm::outs() << "invoking action '" << Subcommand.getName() << "':\n";
+    if (Context.getSelectionRange().isValid()) {
+      SourceRange R = Context.getSelectionRange();
+      llvm::outs() << "  -selection=";
+      R.getBegin().print(llvm::outs(), Context.getSources());
+      llvm::outs() << " -> ";
+      R.getEnd().print(llvm::outs(), Context.getSources());
+      llvm::outs() << "\n";
+    }
+  }
+
+  bool invokeAction(RefactoringActionSubcommand &Subcommand) {
+    // Find a set of matching rules.
+    SmallVector<RefactoringActionRule *, 4> MatchingRules;
+    llvm::StringSet<> MissingOptions;
+
+    bool HasSelection = false;
+    for (const auto &Rule : Subcommand.getActionRules()) {
+      if (Rule->hasSelectionRequirement()) {
+        HasSelection = true;
+        if (Subcommand.getSelection())
+          MatchingRules.push_back(Rule.get());
+        else
+          MissingOptions.insert("selection");
+      }
+      // FIXME (Alex L): Support custom options.
+    }
+    if (MatchingRules.empty()) {
+      llvm::errs() << "error: '" << Subcommand.getName()
+                   << "' can't be invoked with the given arguments:\n";
+      for (const auto &Opt : MissingOptions)
+        llvm::errs() << "  missing '-" << Opt.getKey() << "' option\n";
+      return true;
+    }
+
+    bool HasFailed = false;
+    ClangRefactorConsumer Consumer;
+    if (foreachTranslationUnit(Subcommand, [&](ASTContext &AST) {
+          RefactoringRuleContext Context(AST.getSourceManager());
+          Context.setASTContext(AST);
+
+          auto InvokeRule = [&](RefactoringResultConsumer &Consumer) {
+            logInvocation(Subcommand, Context);
+            for (RefactoringActionRule *Rule : MatchingRules) {
+              if (!Rule->hasSelectionRequirement())
+                continue;
+              Rule->invoke(Consumer, Context);
+              return;
+            }
+            // FIXME (Alex L): If more than one initiation succeeded, then the
+            // rules are ambiguous.
+            llvm_unreachable(
+                "The action must have at least one selection rule");
+          };
+
+          if (HasSelection) {
+            assert(Subcommand.getSelection() && "Missing selection argument?");
+            if (opts::Verbose)
+              Subcommand.getSelection()->print(llvm::outs());
+            auto CustomConsumer =
+                Subcommand.getSelection()->createCustomConsumer();
+            if (Subcommand.getSelection()->forAllRanges(
+                    Context.getSources(), [&](SourceRange R) {
+                      Context.setSelectionRange(R);
+                      InvokeRule(CustomConsumer ? *CustomConsumer : Consumer);
+                    }))
+              HasFailed = true;
+            return;
+          }
+          // FIXME (Alex L): Implement non-selection based invocation path.
+        }))
+      return true;
+    return HasFailed;
+  }
+};
+
+} // end anonymous namespace
+
+int main(int argc, const char **argv) {
+  ClangRefactorTool Tool;
+
+  // FIXME: Use LibTooling's CommonOptions parser when subcommands are supported
+  // by it.
+  cl::HideUnrelatedOptions(opts::CommonRefactorOptions);
+  cl::ParseCommandLineOptions(
+      argc, argv, "Clang-based refactoring tool for C, C++ and Objective-C");
+  cl::PrintOptionValues();
+
+  // Figure out which action is specified by the user. The user must specify
+  // the action using a command-line subcommand, e.g. the invocation
+  // `clang-refactor local-rename` corresponds to the `LocalRename` refactoring
+  // action. All subcommands must have a unique names. This allows us to figure
+  // out which refactoring action should be invoked by looking at the first
+  // subcommand that's enabled by LLVM's command-line parser.
+  auto It = llvm::find_if(
+      Tool.SubCommands,
+      [](const std::unique_ptr<RefactoringActionSubcommand> &SubCommand) {
+        return !!(*SubCommand);
+      });
+  if (It == Tool.SubCommands.end()) {
+    llvm::errs() << "error: no refactoring action given\n";
+    llvm::errs() << "note: the following actions are supported:\n";
+    for (const auto &Subcommand : Tool.SubCommands)
+      llvm::errs().indent(2) << Subcommand->getName() << "\n";
+    return 1;
+  }
+  RefactoringActionSubcommand &ActionCommand = **It;
+
+  ArrayRef<std::string> Sources = ActionCommand.getSources();
+  // When -no-dbs is used, at least one file (TU) must be given to any
+  // subcommand.
+  if (opts::NoDatabases && Sources.empty()) {
+    llvm::errs() << "error: must provide paths to the source files when "
+                    "'-no-dbs' is used\n";
+    return 1;
+  }
+  if (ActionCommand.parseArguments())
+    return 1;
+  if (Tool.invokeAction(ActionCommand))
+    return 1;
+
+  return 0;
+}

cfe/trunk/tools/clang-refactor/TestSupport.h

+//===--- TestSupport.h - Clang-based refactoring tool -----------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief Declares datatypes and routines that are used by test-specific code
+/// in clang-refactor.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_CLANG_REFACTOR_TEST_SUPPORT_H
+#define LLVM_CLANG_TOOLS_CLANG_REFACTOR_TEST_SUPPORT_H
+
+#include "clang/Basic/LLVM.h"
+#include "clang/Basic/SourceLocation.h"
+#include "clang/Tooling/Refactoring/RefactoringResultConsumer.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Support/Error.h"
+#include <map>
+#include <string>
+
+namespace clang {
+
+class SourceManager;
+
+namespace refactor {
+
+/// A source selection range that's specified in a test file using an inline
+/// command in the comment. These commands can take the following forms:
+///
+/// - /*range=*/ will create an empty selection range in the default group
+///   right after the comment.
+/// - /*range a=*/ will create an empty selection range in the 'a' group right
+///   after the comment.
+/// - /*range = +1*/ will create an empty selection range at a location that's
+///   right after the comment with one offset to the column.
+/// - /*range= -> +2:3*/ will create a selection range that starts at the
+///   location right after the comment, and ends at column 3 of the 2nd line
+///   after the line of the starting location.
+///
+/// Clang-refactor will expected all ranges in one test group to produce
+/// identical results.
+struct TestSelectionRange {
+  unsigned Begin, End;
+};
+
+/// A set of test selection ranges specified in one file.
+struct TestSelectionRangesInFile {
+  std::string Filename;
+  struct RangeGroup {
+    std::string Name;
+    SmallVector<TestSelectionRange, 8> Ranges;
+  };
+  std::vector<RangeGroup> GroupedRanges;
+
+  TestSelectionRangesInFile(TestSelectionRangesInFile &&) = default;
+  TestSelectionRangesInFile &operator=(TestSelectionRangesInFile &&) = default;
+
+  bool foreachRange(const SourceManager &SM,
+                    llvm::function_ref<void(SourceRange)> Callback) const;
+
+  std::unique_ptr<tooling::RefactoringResultConsumer> createConsumer() const;
+
+  void dump(llvm::raw_ostream &OS) const;
+};
+
+/// Extracts the grouped selection ranges from the file that's specified in
+/// the -selection=test:<filename> option.
+///
+/// The grouped ranges are specified in comments using the following syntax:
+/// "range" [ group-name ] "=" [ "+" starting-column-offset ] [ "->"
+///                              "+" ending-line-offset ":"
+///                                  ending-column-position ]
+///
+/// The selection range is then computed from this command by taking the ending
+/// location of the comment, and adding 'starting-column-offset' to the column
+/// for that location. That location in turns becomes the whole selection range,
+/// unless 'ending-line-offset' and 'ending-column-position' are specified. If
+/// they are specified, then the ending location of the selection range is
+/// the starting location's line + 'ending-line-offset' and the
+/// 'ending-column-position' column.
+///
+/// All selection ranges in one group are expected to produce the same
+/// refactoring result.
+///
+/// When testing, zero is returned from clang-refactor even when a group
+/// produces an initiation error, which is different from normal invocation
+/// that returns a non-zero value. This is done on purpose, to ensure that group
+/// consistency checks can return non-zero, but still print the output of
+/// the group. So even if a test matches the output of group, it will still fail
+/// because clang-refactor should return zero on exit when the group results are
+/// consistent.
+///
+/// \returns None on failure (errors are emitted to stderr), or a set of
+/// grouped source ranges in the given file otherwise.
+Optional<TestSelectionRangesInFile> findTestSelectionRanges(StringRef Filename);
+
+} // end namespace refactor
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLS_CLANG_REFACTOR_TEST_SUPPORT_H

cfe/trunk/tools/clang-refactor/TestSupport.cpp

+//===--- TestSupport.cpp - Clang-based refactoring tool -------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief This file implements routines that provide refactoring testing
+/// utilities.
+///
+//===----------------------------------------------------------------------===//
+
+#include "TestSupport.h"
+#include "clang/Basic/SourceManager.h"
+#include "clang/Lex/Lexer.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/ErrorOr.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Regex.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace llvm;
+
+namespace clang {
+namespace refactor {
+
+void TestSelectionRangesInFile::dump(raw_ostream &OS) const {
+  for (const auto &Group : GroupedRanges) {
+    OS << "Test selection group '" << Group.Name << "':\n";
+    for (const auto &Range : Group.Ranges) {
+      OS << "  " << Range.Begin << "-" << Range.End << "\n";
+    }
+  }
+}
+
+bool TestSelectionRangesInFile::foreachRange(
+    const SourceManager &SM,
+    llvm::function_ref<void(SourceRange)> Callback) const {
+  const FileEntry *FE = SM.getFileManager().getFile(Filename);
+  FileID FID = FE ? SM.translateFile(FE) : FileID();
+  if (!FE || FID.isInvalid()) {
+    llvm::errs() << "error: -selection=test:" << Filename
+                 << " : given file is not in the target TU";
+    return true;
+  }
+  SourceLocation FileLoc = SM.getLocForStartOfFile(FID);
+  for (const auto &Group : GroupedRanges) {
+    for (const TestSelectionRange &Range : Group.Ranges) {
+      // Translate the offset pair to a true source range.
+      SourceLocation Start =
+          SM.getMacroArgExpandedLocation(FileLoc.getLocWithOffset(Range.Begin));
+      SourceLocation End =
+          SM.getMacroArgExpandedLocation(FileLoc.getLocWithOffset(Range.End));
+      assert(Start.isValid() && End.isValid() && "unexpected invalid range");
+      Callback(SourceRange(Start, End));
+    }
+  }
+  return false;
+}
+
+namespace {
+
+void dumpChanges(const tooling::AtomicChanges &Changes, raw_ostream &OS) {
+  for (const auto &Change : Changes)
+    OS << const_cast<tooling::AtomicChange &>(Change).toYAMLString() << "\n";
+}
+
+bool areChangesSame(const tooling::AtomicChanges &LHS,
+                    const tooling::AtomicChanges &RHS) {
+  if (LHS.size() != RHS.size())
+    return false;
+  for (const auto &I : llvm::zip(LHS, RHS)) {
+    if (!(std::get<0>(I) == std::get<1>(I)))
+      return false;
+  }
+  return true;
+}
+
+bool printRewrittenSources(const tooling::AtomicChanges &Changes,
+                           raw_ostream &OS) {
+  std::set<std::string> Files;
+  for (const auto &Change : Changes)
+    Files.insert(Change.getFilePath());
+  tooling::ApplyChangesSpec Spec;
+  Spec.Cleanup = false;
+  for (const auto &File : Files) {
+    llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> BufferErr =
+        llvm::MemoryBuffer::getFile(File);
+    if (!BufferErr) {
+      llvm::errs() << "failed to open" << File << "\n";
+      return true;
+    }
+    auto Result = tooling::applyAtomicChanges(File, (*BufferErr)->getBuffer(),
+                                              Changes, Spec);
+    if (!Result) {
+      llvm::errs() << toString(Result.takeError());
+      return true;
+    }
+    OS << *Result;
+  }
+  return false;
+}
+
+class TestRefactoringResultConsumer final
+    : public tooling::RefactoringResultConsumer {
+public:
+  TestRefactoringResultConsumer(const TestSelectionRangesInFile &TestRanges)
+      : TestRanges(TestRanges) {
+    Results.push_back({});
+  }
+
+  ~TestRefactoringResultConsumer() {
+    // Ensure all results are checked.
+    for (auto &Group : Results) {
+      for (auto &Result : Group) {
+        if (!Result) {
+          (void)llvm::toString(Result.takeError());
+        }
+      }
+    }
+  }
+
+  void handleError(llvm::Error Err) override { handleResult(std::move(Err)); }
+
+  void handle(tooling::AtomicChanges Changes) override {
+    handleResult(std::move(Changes));
+  }
+
+  void handle(tooling::SymbolOccurrences Occurrences) override {
+    tooling::RefactoringResultConsumer::handle(std::move(Occurrences));
+  }
+
+private:
+  bool handleAllResults();
+
+  void handleResult(Expected<tooling::AtomicChanges> Result) {
+    Results.back().push_back(std::move(Result));
+    size_t GroupIndex = Results.size() - 1;
+    if (Results.back().size() >=
+        TestRanges.GroupedRanges[GroupIndex].Ranges.size()) {
+      ++GroupIndex;
+      if (GroupIndex >= TestRanges.GroupedRanges.size()) {
+        if (handleAllResults())
+          exit(1); // error has occurred.
+        return;
+      }
+      Results.push_back({});
+    }
+  }
+
+  const TestSelectionRangesInFile &TestRanges;
+  std::vector<std::vector<Expected<tooling::AtomicChanges>>> Results;
+};
+
+std::pair<unsigned, unsigned> getLineColumn(StringRef Filename,
+                                            unsigned Offset) {
+  ErrorOr<std::unique_ptr<MemoryBuffer>> ErrOrFile =
+      MemoryBuffer::getFile(Filename);
+  if (!ErrOrFile)
+    return {0, 0};
+  StringRef Source = ErrOrFile.get()->getBuffer();
+  Source = Source.take_front(Offset);
+  size_t LastLine = Source.find_last_of("\r\n");
+  return {Source.count('\n') + 1,
+          (LastLine == StringRef::npos ? Offset : Offset - LastLine) + 1};
+}
+
+} // end anonymous namespace
+
+bool TestRefactoringResultConsumer::handleAllResults() {
+  bool Failed = false;
+  for (auto &Group : llvm::enumerate(Results)) {
+    // All ranges in the group must produce the same result.
+    Optional<tooling::AtomicChanges> CanonicalResult;
+    Optional<std::string> CanonicalErrorMessage;
+    for (auto &I : llvm::enumerate(Group.value())) {
+      Expected<tooling::AtomicChanges> &Result = I.value();
+      std::string ErrorMessage;
+      bool HasResult = !!Result;
+      if (!HasResult) {
+        // FIXME: Handle diagnostic error as well.
+        handleAllErrors(Result.takeError(), [&](StringError &Err) {
+          ErrorMessage = Err.getMessage();
+        });
+      }
+      if (!CanonicalResult && !CanonicalErrorMessage) {
+        if (HasResult)
+          CanonicalResult = std::move(*Result);
+        else
+          CanonicalErrorMessage = std::move(ErrorMessage);
+        continue;
+      }
+
+      // Verify that this result corresponds to the canonical result.
+      if (CanonicalErrorMessage) {
+        // The error messages must match.
+        if (!HasResult && ErrorMessage == *CanonicalErrorMessage)
+          continue;
+      } else {
+        assert(CanonicalResult && "missing canonical result");
+        // The results must match.
+        if (HasResult && areChangesSame(*Result, *CanonicalResult))
+          continue;
+      }
+      Failed = true;
+      // Report the mismatch.
+      std::pair<unsigned, unsigned> LineColumn = getLineColumn(
+          TestRanges.Filename,
+          TestRanges.GroupedRanges[Group.index()].Ranges[I.index()].Begin);
+      llvm::errs()
+          << "error: unexpected refactoring result for range starting at "
+          << LineColumn.first << ':' << LineColumn.second << " in group '"
+          << TestRanges.GroupedRanges[Group.index()].Name << "':\n  ";
+      if (HasResult)
+        llvm::errs() << "valid result";
+      else
+        llvm::errs() << "error '" << ErrorMessage << "'";
+      llvm::errs() << " does not match initial ";
+      if (CanonicalErrorMessage)
+        llvm::errs() << "error '" << *CanonicalErrorMessage << "'\n";
+      else
+        llvm::errs() << "valid result\n";
+      if (HasResult && !CanonicalErrorMessage) {
+        llvm::errs() << "  Expected to Produce:\n";
+        dumpChanges(*CanonicalResult, llvm::errs());
+        llvm::errs() << "  Produced:\n";
+        dumpChanges(*Result, llvm::errs());
+      }
+    }
+
+    // Dump the results:
+    const auto &TestGroup = TestRanges.GroupedRanges[Group.index()];
+    if (!CanonicalResult) {
+      llvm::errs() << TestGroup.Ranges.size() << " '" << TestGroup.Name
+                   << "' results:\n";
+      llvm::errs() << *CanonicalErrorMessage << "\n";
+    } else {
+      llvm::outs() << TestGroup.Ranges.size() << " '" << TestGroup.Name
+                   << "' results:\n";
+      if (printRewrittenSources(*CanonicalResult, llvm::outs()))
+        return true;
+    }
+  }
+  return Failed;
+}
+
+std::unique_ptr<tooling::RefactoringResultConsumer>
+TestSelectionRangesInFile::createConsumer() const {
+  return llvm::make_unique<TestRefactoringResultConsumer>(*this);
+}
+
+/// Adds the \p ColumnOffset to file offset \p Offset, without going past a
+/// newline.
+static unsigned addColumnOffset(StringRef Source, unsigned Offset,
+                                unsigned ColumnOffset) {
+  if (!ColumnOffset)
+    return Offset;
+  StringRef Substr = Source.drop_front(Offset).take_front(ColumnOffset);
+  size_t NewlinePos = Substr.find_first_of("\r\n");
+  return Offset +
+         (NewlinePos == StringRef::npos ? ColumnOffset : (unsigned)NewlinePos);
+}
+
+Optional<TestSelectionRangesInFile>
+findTestSelectionRanges(StringRef Filename) {
+  ErrorOr<std::unique_ptr<MemoryBuffer>> ErrOrFile =
+      MemoryBuffer::getFile(Filename);
+  if (!ErrOrFile) {
+    llvm::errs() << "error: -selection=test:" << Filename
+                 << " : could not open the given file";
+    return None;
+  }
+  StringRef Source = ErrOrFile.get()->getBuffer();
+
+  // FIXME (Alex L): 3rd capture groups for +line:column.
+  // See the doc comment for this function for the explanation of this
+  // syntax.
+  static Regex RangeRegex("range[[:blank:]]*([[:alpha:]_]*)?[[:blank:]]*=[[:"
+                          "blank:]]*(\\+[[:digit:]]+)?");
+
+  std::map<std::string, SmallVector<TestSelectionRange, 8>> GroupedRanges;
+
+  LangOptions LangOpts;
+  LangOpts.CPlusPlus = 1;
+  LangOpts.CPlusPlus11 = 1;
+  Lexer Lex(SourceLocation::getFromRawEncoding(0), LangOpts, Source.begin(),
+            Source.begin(), Source.end());
+  Lex.SetCommentRetentionState(true);
+  Token Tok;
+  for (Lex.LexFromRawLexer(Tok); Tok.isNot(tok::eof);
+       Lex.LexFromRawLexer(Tok)) {
+    if (Tok.isNot(tok::comment))
+      continue;
+    StringRef Comment =
+        Source.substr(Tok.getLocation().getRawEncoding(), Tok.getLength());
+    SmallVector<StringRef, 4> Matches;
+    // Allow CHECK: comments to contain range= commands.
+    if (!RangeRegex.match(Comment, &Matches) || Comment.contains("CHECK")) {
+      // Try to detect mistyped 'range:' comments to ensure tests don't miss
+      // anything.
+      if (Comment.contains_lower("range") && Comment.contains("=") &&
+          !Comment.contains_lower("run") && !Comment.contains("CHECK")) {
+        llvm::errs() << "error: suspicious comment '" << Comment
+                     << "' that "
+                        "resembles the range command found\n";
+        llvm::errs() << "note: please reword if this isn't a range command\n";
+        return None;
+      }
+      continue;
+    }
+    unsigned Offset = Tok.getEndLoc().getRawEncoding();
+    unsigned ColumnOffset = 0;
+    if (!Matches[2].empty()) {
+      // Don't forget to drop the '+'!
+      if (Matches[2].drop_front().getAsInteger(10, ColumnOffset))
+        assert(false && "regex should have produced a number");
+    }
+    // FIXME (Alex L): Support true ranges.
+    Offset = addColumnOffset(Source, Offset, ColumnOffset);
+    TestSelectionRange Range = {Offset, Offset};
+    auto It = GroupedRanges.insert(std::make_pair(
+        Matches[1].str(), SmallVector<TestSelectionRange, 8>{Range}));
+    if (!It.second)
+      It.first->second.push_back(Range);
+  }
+  if (GroupedRanges.empty()) {
+    llvm::errs() << "error: -selection=test:" << Filename
+                 << ": no 'range' commands";
+    return None;
+  }
+
+  TestSelectionRangesInFile TestRanges = {Filename.str(), {}};
+  for (auto &Group : GroupedRanges)
+    TestRanges.GroupedRanges.push_back({Group.first, std::move(Group.second)});
+  return std::move(TestRanges);
+}
+
+} // end namespace refactor
+} // end namespace clang

cfe/trunk/unittests/Tooling/RefactoringActionRulesTest.cpp

createReplacements(const std::unique_ptr<RefactoringActionRule> &Rule,
 
   Consumer C;
   Rule->invoke(C, Context);
   return std::move(*C.Result);
 }
 
 TEST_F(RefactoringActionRulesTest, MyFirstRefactoringRule) {
   auto ReplaceAWithB =
-      [](std::pair<selection::SourceSelectionRange, int> Selection)
+      [](const RefactoringRuleContext &,
+         std::pair<selection::SourceSelectionRange, int> Selection)
       -> Expected<AtomicChanges> {
     const SourceManager &SM = Selection.first.getSources();
     SourceLocation Loc = Selection.first.getRange().getBegin().getLocWithOffset(
         Selection.second);
     AtomicChange Change(SM, Loc);
     llvm::Error E = Change.replace(SM, Loc, 1, "b");
     if (E)
       return std::move(E);
     return AtomicChanges{Change};
   };
   class SelectionRequirement : public selection::Requirement {
   public:
     std::pair<selection::SourceSelectionRange, int>
-    evaluateSelection(selection::SourceSelectionRange Selection) const {
+    evaluateSelection(const RefactoringRuleContext &,
+                      selection::SourceSelectionRange Selection) const {
       return std::make_pair(Selection, 20);
     }
   };
   auto Rule = createRefactoringRule(ReplaceAWithB,
                                     requiredSelection(SelectionRequirement()));
 
   // When the requirements are satisifed, the rule's function must be invoked.
   {
llvm::handleAllErrors(
         ErrorOrResult.takeError(),
         [&](llvm::StringError &Error) { Message = Error.getMessage(); });
     EXPECT_EQ(Message, "refactoring action can't be initiated with the "
                        "specified selection range");
   }
 }
 
 TEST_F(RefactoringActionRulesTest, ReturnError) {
-  Expected<AtomicChanges> (*Func)(selection::SourceSelectionRange) =
-      [](selection::SourceSelectionRange) -> Expected<AtomicChanges> {
+  Expected<AtomicChanges> (*Func)(const RefactoringRuleContext &,
+                                  selection::SourceSelectionRange) =
+      [](const RefactoringRuleContext &,
+         selection::SourceSelectionRange) -> Expected<AtomicChanges> {
     return llvm::make_error<llvm::StringError>(
         "Error", llvm::make_error_code(llvm::errc::invalid_argument));
   };
   auto Rule = createRefactoringRule(
       Func, requiredSelection(
                 selection::identity<selection::SourceSelectionRange>()));
 
   RefactoringRuleContext RefContext(Context.Sources);
TEST_F(RefactoringActionRulesTest, ReturnError) {
   EXPECT_EQ(Message, "Error");
 }
 
 TEST_F(RefactoringActionRulesTest, ReturnInitiationDiagnostic) {
   RefactoringRuleContext RefContext(Context.Sources);
   class SelectionRequirement : public selection::Requirement {
   public:
     Expected<Optional<int>>
-    evaluateSelection(selection::SourceSelectionRange Selection) const {
+    evaluateSelection(const RefactoringRuleContext &,
+                      selection::SourceSelectionRange Selection) const {
       return llvm::make_error<llvm::StringError>(
           "bad selection", llvm::make_error_code(llvm::errc::invalid_argument));
     }
   };
   auto Rule = createRefactoringRule(
-      [](int) -> Expected<AtomicChanges> {
+      [](const RefactoringRuleContext &, int) -> Expected<AtomicChanges> {
         llvm::report_fatal_error("Should not run!");
       },
       requiredSelection(SelectionRequirement()));
 
   SourceLocation Cursor =
       Context.Sources.getLocForStartOfFile(Context.Sources.getMainFileID());
   RefContext.setSelectionRange({Cursor, Cursor});
   Expected<AtomicChanges> Result = createReplacements(Rule, RefContext);
Optional<SymbolOccurrences> findOccurrences(RefactoringActionRule &Rule,
 
   Consumer C;
   Rule.invoke(C, Context);
   return std::move(C.Result);
 }
 
 TEST_F(RefactoringActionRulesTest, ReturnSymbolOccurrences) {
   auto Rule = createRefactoringRule(
-      [](selection::SourceSelectionRange Selection)
+      [](const RefactoringRuleContext &,
+         selection::SourceSelectionRange Selection)
           -> Expected<SymbolOccurrences> {
         SymbolOccurrences Occurrences;
         Occurrences.push_back(SymbolOccurrence(
             SymbolName("test"), SymbolOccurrence::MatchingSymbol,
             Selection.getRange().getBegin()));
         return std::move(Occurrences);
       },
       requiredSelection(