[clang-refactor] introducing clang-refactor

[omtcyfz]

Summary

Adding clang-refactor with a dummy template refactoring module.

The tool itself doesn't have to be perfect in the first iteration, so I would be really happy to push this one early enough so that everyone could start building modules on top of it. As soon as this one is landed more refactoring and cleanup patches are welcome. This also isn't about "the one true way how clang-refactor has to be designed". Thus said, consider this version of clang-refactor to be highly experimental.

Clang-refactor design doc can be found here: https://docs.google.com/document/d/1w9IkR0_Gqmd5w4CZ2t_ZDZrNLYVirQPyMS41533HQZE/edit?usp=sharing

UPD: This diff was stalled for a while, clang-refactor was introduced in https://reviews.llvm.org/D36574 patch by araphman. Removing everyone from subscribers and reviewers for convenience.

[ioeric]

• You mentioned a design doc in the summary; maybe also include a link to it?
• It would make the review easier if you could separate the migration of clang-rename into another patch...I think clang-refactor is really the interesting part here. And maybe also add a small dummy sub-module as an example demonstrating how to add a new sub-module, which I believe will be helpful for future developers. (See also tool-template which exists for the same purpose.)

TemplatedClassFunction.cpp

1 ↗

(On Diff #70163)

What is this file for?

[omtcyfz]

In D24192#532981, @ioeric wrote:

• You mentioned a design doc in the summary; maybe also include a link to it?

Done.

• It would make the review easier if you could separate the migration of clang-rename into another patch...I think clang-refactor is really the interesting part here. And maybe also add a small dummy sub-module as an example demonstrating how to add a new sub-module, which I believe will be helpful for future developers. (See also tool-template which exists for the same purpose.)

I am not against it, but I do not think that an empty tool, which just introduces the binary without any code would be reasonable. This is basically why I wanted it to be all-in-place.

The whole changelist doesn't introduce anything really new, so I assume it might be fine. But if you are not okay with it I can split.

TemplatedClassFunction.cpp

1 ↗

(On Diff #70163)

This is a test. It is the reason clang-refactor rename fails :)

[omtcyfz]

Removed "clang-refactor (as opposed to clang-rename) now uses LLVM policy about brackets around single line statements in if/while/..." change to make the review a bit easier.

[omtcyfz]

Revert diff, as the last one "deletes and creates" files instead of "moving and changing them" in the filesystem.

[omtcyfz]

In D24192#532981, @ioeric wrote:

• It would make the review easier if you could separate the migration of clang-rename into another patch...

Another point is that if I try to separate the migration - what do I do about USREngine? USREngine is basically the core of clang-refactor at the moment and I can't detach it from clang-rename at the same time.

[ioeric]

In D24192#533174, @omtcyfz wrote:

In D24192#532981, @ioeric wrote:

• It would make the review easier if you could separate the migration of clang-rename into another patch...

Another point is that if I try to separate the migration - what do I do about USREngine? USREngine is basically the core of clang-refactor at the moment and I can't detach it from clang-rename at the same time.

I'm not sure why USREngine is the core of clang-refactor. It seems to me that USREngine is more closely tied to clang-rename than to clang-refactor. At least USREngine is not essential to all refactor tools, and it is more like a library that sub-modules can use.

[omtcyfz]

In D24192#533198, @ioeric wrote:

In D24192#533174, @omtcyfz wrote:

In D24192#532981, @ioeric wrote:

• It would make the review easier if you could separate the migration of clang-rename into another patch...

Another point is that if I try to separate the migration - what do I do about USREngine? USREngine is basically the core of clang-refactor at the moment and I can't detach it from clang-rename at the same time.

I'm not sure why USREngine is the core of clang-refactor. It seems to me that USREngine is more closely tied to clang-rename than to clang-refactor. At least USREngine is not essential to all refactor tools, and it is more like a library that sub-modules can use.

It is essential to all of the tools I wrote about in design doc.

Well, are you proposing to create an "empty" clang-refactor binary in one patch and adding meaningful code in the other? I am not sure if just creating clang-refactor/driver/Driver.cpp with main, which doesn't do anything is a good idea, but if you think it is - I'll do that.

[ioeric]

In D24192#533082, @omtcyfz wrote:

In D24192#532981, @ioeric wrote:

• You mentioned a design doc in the summary; maybe also include a link to it?

Done.

• It would make the review easier if you could separate the migration of clang-rename into another patch...I think clang-refactor is really the interesting part here. And maybe also add a small dummy sub-module as an example demonstrating how to add a new sub-module, which I believe will be helpful for future developers. (See also tool-template which exists for the same purpose.)

I am not against it, but I do not think that an empty tool, which just introduces the binary without any code would be reasonable. This is basically why I wanted it to be all-in-place.

The whole changelist doesn't introduce anything really new, so I assume it might be fine. But if you are not okay with it I can split.

Don't worry about a patch being small. Reviewers like small patches :)

The reason that I suggested a dummy sub-tool is to lower the bar for developers, especially those who have never developed a clang tool, so that they can easily figure out how to set up a clang-refactor sub-tool.

For a "swiss-army knife" tool like clang-refactor, I would worry more about the extendibility of the infrastructure instead of functionality at this stage. At least, I would expect it to be much easier to create a refactor tool under clang-refactor than creating a standalone clang tool. For example, it would be really nice if there is a tool base that a sub-module can derive from so that I don't have to go through the whole process of creating a clang tool and setting up unit/lit test environment :)

[ioeric]

In D24192#533200, @omtcyfz wrote:

In D24192#533198, @ioeric wrote:

In D24192#533174, @omtcyfz wrote:

In D24192#532981, @ioeric wrote:

• It would make the review easier if you could separate the migration of clang-rename into another patch...

Another point is that if I try to separate the migration - what do I do about USREngine? USREngine is basically the core of clang-refactor at the moment and I can't detach it from clang-rename at the same time.

I'm not sure why USREngine is the core of clang-refactor. It seems to me that USREngine is more closely tied to clang-rename than to clang-refactor. At least USREngine is not essential to all refactor tools, and it is more like a library that sub-modules can use.

It is essential to all of the tools I wrote about in design doc.

Well, are you proposing to create an "empty" clang-refactor binary in one patch and adding meaningful code in the other? I am not sure if just creating clang-refactor/driver/Driver.cpp with main, which doesn't do anything is a good idea, but if you think it is - I'll do that.

It was not trivial to me why USREngine is so important to those tools. You might want to address that in the design doc as well. And given the weight USREngine carries in clang-refactor as you suggested, I think it deserves an more detailed introduction in the design doc.

[alexander-shaposhnikov]

clang-refactor/driver/Rename.h

192 ↗

(On Diff #70194)

if i am not mistaken if the file has not been modified this will print smth like
"invalid source location" or similar. Pls, double check me.

[omtcyfz]

In D24192#533233, @ioeric wrote:

It was not trivial to me why USREngine is so important to those tools. You might want to address that in the design doc as well. And given the weight USREngine carries in clang-refactor as you suggested, I think it deserves an more detailed introduction in the design doc.

Yes, I agree, it might be a little confusing. The only mention I have is:

The most important facility would allow lookup of symbol definitions and usages (probably, based on Unified Symbol Resolutions, USR)

But it doesn't get into too much details anyway. However, the idea of the design doc was not to focus on implementation details too much as they are a subject to change.

Thus, I'm not sure whether I should elaborate details of USREngine there.

clang-refactor/driver/Rename.h

192 ↗

(On Diff #70194)

Sorry, can you please elaborate?

Also, this might be unrelated to this exact patch, but if there is an issue it's good to fix it in another patch. Basically, this is not new code, it's been there in clang-rename.

[omtcyfz]

In D24192#533220, @ioeric wrote:

Don't worry about a patch being small. Reviewers like small patches :)

The reason that I suggested a dummy sub-tool is to lower the bar for developers, especially those who have never developed a clang tool, so that they can easily figure out how to set up a clang-refactor sub-tool.

For a "swiss-army knife" tool like clang-refactor, I would worry more about the extendibility of the infrastructure instead of functionality at this stage. At least, I would expect it to be much easier to create a refactor tool under clang-refactor than creating a standalone clang tool. For example, it would be really nice if there is a tool base that a sub-module can derive from so that I don't have to go through the whole process of creating a clang tool and setting up unit/lit test environment :)

Yes, you are absolutely right. This patch is just a mess, because I tried to push everything at once while most of the stuff is not related to clang-refactor part at all.

I pushed 3 very basic cleanup patches (namely rL280638, rL280639 and rL280640), which don't really need reviewing (and also for the sake of development speed). And I also kept clang-rename improvements separate in D24224, which is up for review.

I'll update this patch tomorrow.

What I'm about to do after is to either:

1. Make two patches, one of which would basically create "empty" clang-refactor and introduce dummy submodule to show how it works (as you proposed), and the other would merge clang-rename as a submodule.
2. Make one patch, which both creates clang-refactor binary and merges clang-rename as a submodule.

The first variant seems better to me while I see a couple of issues with that, which makes sticking to the second variant a bit less challenging. We can discuss it tomorr..today and come up with a solution.

Thank you very much for your comments, Eric!

[omtcyfz]

Removed whole clang-rename part, only making this patch clang-refactor specific.

[omtcyfz]

Bringing results of an offline discussion with Eric (@ioeric) live.

Eric's point was that this patch should only care about clang-refactor and introduce changes directly related to creating clang-rename. clang-rename and all other tools migration can be done later, which is also good in terms of this patch not growing too large.

Another point was that it should be easier if there would be a class, from which refactoring tools would derive and use it as an interface.

Advantages are as follows:

• It would be easier to implement multi-TU support as the tools wouldn't simply care about that part while delegating the "hard work" to the common part.
• It would be easier to implement new tools, because there will be a slightly simpler interface.
• It would be easier to setup unit tests and other routine, because right now all tools have to implement basic routine for unit tests, for example, which is just copied from the other tools' unit tests with slight changes. Delegating that to the common interface is also nice.

The current diff isn't final, but I wanted to bring it to the reviews, because it gives an overview of what I am doing and if there are any high-level comments to address it's better to get them sooner than later.

[hokein]

Eric's point was that this patch should only care about clang-refactor and introduce changes directly related to creating clang-rename. clang-rename and all other tools migration can be done later, which is also good in terms of this patch not growing too large.

I'm +1 on making this patch implement a skeleton of clang-refactor, which makes it small, and much easier for review. The patch looks much better now.

clang-refactor/driver/Driver.cpp

11

clang-tidy?

38

s/clan/clang

47

Any reason implementing the command-line options and not using the cl::opt here?

clang-refactor/driver/ModuleManager.h

24 ↗

(On Diff #70373)

StringRef.

clang-refactor/modules/core/RefactoringModule.h

43 ↗

(On Diff #70373)

s/wouold/would/

54 ↗

(On Diff #70373)

"It it" doesn't make sense here.

89 ↗

(On Diff #70373)

I'd make this interface a virtual function too, and provide this default implementation.

128 ↗

(On Diff #70373)

Does the panic mean something like exit(1)? But from the interface, it looks like returning an error code, right?

146 ↗

(On Diff #70373)

A further thought:

applyReplacement is more likely a postprocess step of handleTranslationUnit, what if some clang-refactor subtools just want to dump the results only? I'd prefer to rename it to OnEndOfHandlingTranslationUnit. I'm open to hear better suggestions.

148 ↗

(On Diff #70373)

Use "= default".

150 ↗

(On Diff #70373)

You can return a StringRef here?

152 ↗

(On Diff #70373)

The same.

clang-refactor/modules/template/TemplateModule.h

15 ↗

(On Diff #70373)

Not needed.

20 ↗

(On Diff #70373)

It'd be clear to add a small paragraph describing that this is a template module which is only used to lower the "barrier to entry" for writing clang-refactor subtool.

[omtcyfz]

Addressing a round of comments.

[omtcyfz]

Thank you for reviewing, @hokein!

Also, please note that this is not a final version, the interface will change a lot in the upcoming diffs.

clang-refactor/driver/Driver.cpp

47

I might be mistaken, but it'd require parsing whole option sequence, which should be delegated to the submodules.

E.g. "help" should be only called if the the tool is invoked as clang-refactor --help, but if I parse all options I'd also invoke "help" while having "clang-refactor rename --help" for example. At least that's what I have been thinking.

clang-refactor/modules/core/RefactoringModule.h

89 ↗

(On Diff #70373)

I'll update the interface soonish.

Had an offline discussion with Alex about the interface and we came up to a conclusion that the interface should be way more strict. This one is just a general idea of how it might be implemented.

Thus, run shouldn't be overridden in my opinion, but I'll update the diff and make some high-level comments on that later.

128 ↗

(On Diff #70373)

1. Well, no. I believe that the error should be thrown to the run and "printed out" there.
2. Also, not that we're aiming for a multi-TU multi-threaded architecture in the end (although it doesn't make sense with the current interface, but again, I'll update it later).

146 ↗

(On Diff #70373)

/* comment about multi-TU multi-threaded stuff */

Suppose a refactoring can't be applied to some translation units. Say, "rename" tool encountered name conflict in any TU. In this case the tool should fail and not process the last step.

If your comment is about naming only, I'll change it, too.

clang-refactor/modules/template/TemplateModule.h

15 ↗

(On Diff #70373)

Deleting TemplateModule.

20 ↗

(On Diff #70373)

Deleting TemplateModule.

[curdeius]

For the moment, just a few nitty-gritty comments inline.
What I miss here is (as already pointed by someone) an example on how to write a new module, register it etc.

clang-refactor/driver/Driver.cpp

42

Simpler:

std::string Invocation = argv[0] + (" " + Command);

clang-refactor/driver/ModuleManager.cpp

22–24 ↗

(On Diff #70397)

Using find and then operator[] makes the search twice. IMO, it would be better to avoid that by:

auto it = CommandToModuleID.find(Command);
if (it != CommandToModuleID.end()) {
  return RegisteredModules[*it]->run(argc, argv);
}

clang-refactor/driver/ModuleManager.h

13–19 ↗

(On Diff #70397)

clang-format includes (and make it a single block)?

21 ↗

(On Diff #70397)

using namespace in header?!

clang-refactor/modules/core/RefactoringModule.h

36 ↗

(On Diff #70397)

s/infromation/information

56 ↗

(On Diff #70397)

s/occurance/occurrence

72 ↗

(On Diff #70397)

s/temprorary/temporary

112 ↗

(On Diff #70397)

s/regiestering/registering

[alexfh]

Kirill, first, thank you for working on this!

As discussed offline, the RefactoringModule interface should explicitly define:

• separation of execution stages for each check:
  1. preparation or execution planning - should figure out the set of affected translation units and what should be done for each of them
  2. processing of affected translation units - ideally, this should be done in parallel, separate process per TU, and the interface should not prevent this
  3. application of replacements or error reporting, if stage 2 detected that execution of refactoring is not possible - looks like this is not refactoring-specific and can be done by the clang-refactor framework
• state transferred between stages (since they should allow for independent execution, potentially in multiple processes, so data should be in a serializable form)
• data and facilities available to each stage (e.g. cross-reference index, AST of a certain translation unit, ability to register PPCallbacks, interface to pass data to the next stage, report results or errors)
• basic interfaces for the facilities mentioned above.

A failure to define these aspects would complicate introduction of multi-TU support and require significant changes to refactoring tools to follow a stricter interface.

[alexfh]

clang-refactor/driver/Driver.cpp

42

(llvm::Twine(argv[0]) + " " + ...).str()

clang-refactor/driver/ModuleManager.h

21 ↗

(On Diff #70397)

If you need StringRef, ArrayRef etc. in clang namespace, just include "clang/Basic/LLVM.h".

clang-refactor/modules/core/RefactoringModule.h

91 ↗

(On Diff #70397)

I believe, run should be external to the module. Its implementation might be different depending on whether we're just running all stages sequentially or trying to parallelize processing of different translation units.

148 ↗

(On Diff #70397)

This should be possible to implement in a refactoring-independent way in clang-refactor itself. Or do you see any issues with this?

[omtcyfz]

Addressing few comments.

Major improvements on the way.

[omtcyfz]

clang-refactor/driver/ModuleManager.cpp

22–24 ↗

(On Diff #70397)

The search is O(1) anyway, but the code itself probably becomes more reasonable, thanks.

clang-refactor/driver/ModuleManager.h

13–19 ↗

(On Diff #70397)

I like to separate includes into three groups (LLVM includes, STL includes, subproject includes), which I think is not very bad. Haven't seen any policy prohibiting it. Feel free to elaborate, though.

Sorted the includes within the second group.

clang-refactor/modules/core/RefactoringModule.h

148 ↗

(On Diff #70397)

Nope, totally agree.

[curdeius]

clang-refactor/driver/ModuleManager.h

14–20 ↗

(On Diff #70682)

I thought that idea behind sorting includes using clang-format is to avoid handling groups and order manually.
I don't think that there is any policy prohibiting separating includes into groups, but AFAIK, there is one that says that STL includes should be the last (you should include in order from the most specific to the most generic, i.e. subproject, clang, llvm, STL).

[omtcyfz]

Slightly improve the interface.

Patch is still not complete, though.

[omtcyfz]

The diff doesn't really do much right now, but it kind of gives the idea of what's happening with clang-refactor.

[ioeric]

clang-refactor/driver/ModuleManager.h

14–20 ↗

(On Diff #70682)

Fortunately, clang-format is smart enough to categorize #include groups (e.g. LLVM includes, STL includes, main header etc). We actually encourage people to combine #includes groups into one block and let clang-format handle the categorization (fyi: clang-format only sort includes within a block). The point is to free you from worrying about the formatting.

[arphaman]

clang-refactor/core/SymbolIndexClient.h

31

End namespace clang?

[arphaman]

clang-refactor/driver/Driver.cpp

36

Is there a particular reason why you don't return a value that's returned by Manager.dispatch here? It seems odd to me that Manager.dispatch returns an integer value that's not actually used here.

[ioeric]

clang-refactor/driver/Driver.cpp

36

BTW, isn't argv[1] pointing to a temp string above?

[omtcyfz]

No longer relevant.

CMakeLists.txt

 add_subdirectory(clang-apply-replacements)
 add_subdirectory(clang-rename)
+add_subdirectory(clang-refactor)
 add_subdirectory(clang-reorder-fields)
 add_subdirectory(modularize)
 if(CLANG_ENABLE_STATIC_ANALYZER)

 if( CLANG_TOOLS_EXTRA_INCLUDE_DOCS )
   add_subdirectory(docs)
 endif()
-

clang-refactor/CMakeLists.txt

+include_directories(core)
+
+add_subdirectory(core)
+add_subdirectory(driver)

clang-refactor/core/BasicExecutor.h

+//===--- BasicExecutor.h - clang-refactor -----------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_BASIC_EXECUTOR_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_BASIC_EXECUTOR_H
+
+#include "Executor.h"
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace refactor {
+
+class BasicExecutor : public Executor {
+public:
+  void addTask(StringRef TU, const RefactoringOptions &Opts);
+
+  llvm::Error runTasks();
+private:
+  std::vector<std::pair<std::string, RefactoringOptions>> Tasks;
+};
+
+} // end namespace refactor
+} // end namespace refactor
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_BASIC_EXECUTOR

clang-refactor/core/BasicExecutor.cpp

+//===--- BasicExecutor.cpp - clang-refactor ---------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "BasicExecutor.h"
+
+namespace clang {
+namespace refactor {
+
+void BasicExecutor::addTask(StringRef TU,
+                            const RefactoringOptions &Opts) {
+  Tasks.push_back({TU, Opts});
+}
+
+llvm::Error BasicExecutor::runTasks() {
+  llvm::Error Err;
+  for (const auto &Task : Tasks) {
+    // FIXME: Actually do stuff.
+    (void)Task;
+  }
+  return Err;
+}
+
+} // end namespace refactor
+} // end namespace refactor

clang-refactor/core/CMakeLists.txt

+add_clang_library(clangRefactorCore
+  RefactoringContext.cpp
+  RefactoringModule.cpp
+  BasicExecutor.cpp
+  ModuleManager.cpp
+  ModuleEntryPoint.cpp
+
+  LINK_LIBS
+  clangAST
+  clangASTMatchers
+  clangBasic
+  clangFrontend
+  clangLex
+  clangTooling
+  clangToolingCore
+  )

clang-refactor/core/Executor.h

+//===--- Executor.h - clang-refactor ----------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_EXECUTOR_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_EXECUTOR_H
+
+#include "RefactoringOptions.h"
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace refactor {
+
+class Executor {
+public:
+  virtual void addTask(StringRef TU, const RefactoringOptions &Opts) = 0;
+  virtual llvm::Error runTasks() = 0;
+  virtual ~Executor() = default;
+};
+
+} // end namespace refactor
+} // end namespace refactor
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_EXECUTOR_H

clang-refactor/core/ModuleEntryPoint.h

+//===--- ModuleEntryPoint.h - clang-refactor --------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_MODULE_ENTRY_POINT_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_MODULE_ENTRY_POINT_H
+
+#include "BasicExecutor.h"
+#include "RefactoringModule.h"
+#include "RefactoringContext.h"
+#include "SymbolIndexClient.h"
+#include "llvm/Support/Error.h"
+
+namespace clang {
+namespace refactor {
+
+class RefactoringModule;
+llvm::Error EntryPoint(std::unique_ptr<RefactoringModule> &Module, int argc,
+                       const char **argv);
+
+} // end namespace refactor
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_MODULE_ENTRY_POINT

clang-refactor/core/ModuleEntryPoint.cpp

+//===--- ModuleEntryPoint.cpp - clang-refactor ------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ModuleEntryPoint.h"
+
+namespace clang {
+namespace refactor {
+
+llvm::Error EntryPoint(std::unique_ptr<RefactoringModule> &Module,
+                              int argc, const char **argv) {
+  llvm::Error Err;
+
+  Module->registerOptions();
+
+  // FIXME: Create RefactoringOptions from CLI arguments.
+  RefactoringOptions CommandLineOptions;
+
+  SymbolIndexClient IndexClient;
+  BasicExecutor Exec;
+
+  RefactoringContext Ctx(&IndexClient, &Exec);
+
+  RefactoringOptions PlanExecutionOptions =
+      Module->planExecution(Ctx, CommandLineOptions);
+
+  Exec.runTasks();
+
+  return Err;
+}
+
+} // end namespace refactor
+} // end namespace clang

clang-refactor/core/ModuleManager.h

+//===--- ModuleManager.h - clang-refactor -----------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_MODULE_MANAGER_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_MODULE_MANAGER_H
+
+#include "ModuleEntryPoint.h"
+#include "BasicExecutor.h"
+#include "RefactoringModule.h"
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/StringRef.h"
+#include <map>
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace refactor {
+
+class ModuleManager {
+public:
+  void addModule(StringRef Command, std::unique_ptr<RefactoringModule> Module);
+
+  int dispatch(StringRef Command, int argc, const char **argv);
+
+  std::string getTopLevelHelp();
+
+private:
+  const std::string HelpHead = "USAGE: clang-refactor [subcommand] [options] "
+                               "<source0> [... <sourceN>]\n"
+                               "\n"
+                               "Subcommands:\n";
+
+  std::map<std::string, std::unique_ptr<RefactoringModule>>
+      CommandToModule;
+};
+
+} // end namespace refactor
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_MODULE_MANAGER_H

clang-refactor/core/ModuleManager.cpp

+//===--- ModuleManager.cpp - clang-refactor ---------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ModuleManager.h"
+
+namespace clang {
+namespace refactor {
+
+void ModuleManager::addModule(StringRef Command,
+                              std::unique_ptr<RefactoringModule> Module) {
+  CommandToModule[Command] = std::move(Module);
+}
+
+int ModuleManager::dispatch(StringRef Command, int argc,
+                                    const char **argv) {
+  auto it = CommandToModule.find(Command);
+  if (it != CommandToModule.end()) {
+    EntryPoint(it->second, argc, argv);
+  }
+  return 1;
+}
+
+std::string ModuleManager::getTopLevelHelp() {
+  std::string Result = HelpHead;
+  for (const auto &CommandToModulePair : CommandToModule) {
+    Result += std::string(CommandToModulePair.second->getModuleName());
+  }
+  return Result;
+}
+
+} // end namespace refactor
+} // end namespace clang

clang-refactor/core/RefactoringContext.h

+//===--- RefactoringContext.h - clang-refactor ------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_CONTEXT_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_CONTEXT_H
+
+#include "BasicExecutor.h"
+#include "ModuleEntryPoint.h"
+#include "SymbolIndexClient.h"
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/StringRef.h"
+#include <functional>
+#include <map>
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace refactor {
+
+class RefactoringContext {
+public:
+  RefactoringContext(SymbolIndexClient *IndexClient, Executor *Exec);
+
+  void parseAST(/* ... */);
+
+private:
+  SymbolIndexClient *IndexClient;
+  Executor *Exec;
+  // FIXME: What else?
+  //        File?
+  //        ...?
+};
+
+} // end namespace refactor
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_CONTEXT_H

clang-refactor/core/RefactoringContext.cpp

+//===--- RefactoringContext.cpp - clang-refactor ----------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "RefactoringContext.h"
+
+namespace clang {
+namespace refactor {
+
+RefactoringContext::RefactoringContext(SymbolIndexClient *IndexClient,
+                                       Executor *Exec)
+    : IndexClient(IndexClient), Exec(Exec) {}
+
+void RefactoringContext::parseAST(/* ... */) {}
+
+} // end namespace refactor
+} // end namespace clang

clang-refactor/core/RefactoringModule.h

+//===--- RefactoringModule.h - clang-refactor -------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_MODULE_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_MODULE_H
+
+#include "Executor.h"
+#include "RefactoringContext.h"
+#include "RefactoringOptions.h"
+#include "SymbolIndexClient.h"
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/LLVM.h"
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/Refactoring.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/ErrorOr.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Signals.h"
+#include <string>
+
+namespace clang {
+namespace refactor {
+
+class RefactoringContext;
+
+class RefactoringModule {
+public:
+  RefactoringModule(StringRef ModuleName, StringRef ModuleDescription);
+
+  //===--------------------------------------------------------------------===//
+  // Pre-stage I: register CLI options.
+  //===--------------------------------------------------------------------===//
+  virtual llvm::cl::OptionCategory registerOptions() = 0;
+  //===--------------------------------------------------------------------===//
+
+  //===--------------------------------------------------------------------===//
+  // Stage I: Plan execution.
+  //
+  // FIXME: Describe current stage.
+  //
+  // Input: Options retrieved from command line arguments, ASTContext for
+  // translation unit, to which <source0> belongs to, cross-reference symbol
+  // index, executor.
+  //
+  // Ouput: Submodule-specific information for independent translation unit
+  // processing.
+  //
+  //===--------------------------------------------------------------------===//
+  virtual RefactoringOptions planExecution(const RefactoringContext &Ctx,
+                                           const RefactoringOptions &Opts) = 0;
+  //===--------------------------------------------------------------------===//
+
+  //===--------------------------------------------------------------------===//
+  // Stage II: Running on a single translation unit.
+  //
+  // FIXME: Describe current stage.
+  //
+  // Input: Compiler instance, translation unit main file.
+  //
+  // Output: list of replacements and diagnostics.
+  //
+  // Has access to: AST of given translation unit.
+  //
+  //===--------------------------------------------------------------------===//
+  virtual std::unique_ptr<ASTConsumer>
+  createASTConsumer(CompilerInstance &CI, StringRef InputFile,
+                    const RefactoringOptions &Opts) = 0;
+  //===--------------------------------------------------------------------===//
+
+  StringRef getModuleName();
+  StringRef getModuleDescription();
+
+  virtual ~RefactoringModule();
+
+private:
+  // ModuleName is the lowercase submodule name, which will be passed to
+  // clang-refactor to invoke the submodule tool.
+  const std::string ModuleName;
+
+  // ModuleDescription will appear as the module description upon calling
+  // $ clang-refactor --help
+  const std::string ModuleDescription;
+};
+
+} // end namespace refactor
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_MODULE_H

clang-refactor/core/RefactoringModule.cpp

+//===--- RefactoringModule.cpp - clang-refactor -----------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "RefactoringModule.h"
+
+namespace clang {
+namespace refactor {
+
+RefactoringModule::RefactoringModule(StringRef ModuleName,
+                                     StringRef ModuleDescription)
+    : ModuleName(ModuleName), ModuleDescription(ModuleDescription) {}
+
+StringRef RefactoringModule::getModuleName() { return ModuleName; }
+
+StringRef RefactoringModule::getModuleDescription() {
+  return ModuleDescription;
+}
+
+RefactoringModule::~RefactoringModule() = default;
+
+} // end namespace refactor
+} // end namespace clang

clang-refactor/core/RefactoringOptions.h

+//===--- RefactoringOptions.h - clang-refactor ------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_OPTIONS_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_OPTIONS_H
+
+#include "llvm/ADT/StringMap.h"
+#include <string>
+
+namespace clang {
+namespace refactor {
+
+// FIXME: Once it's not enough, implement <class RefactoringOptions>.
+typedef llvm::StringMap<std::string> RefactoringOptions;
+
+} // end namespace refactor
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_REFACTORING_OPTIONS_H

clang-refactor/core/SymbolIndexClient.h

+//===--- SymbolIndexClient.h - clang-refactor -------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_SYMBOL_INDEX_CLIENT_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_SYMBOL_INDEX_CLIENT_H
+
+#include <vector>
+#include <string>
+
+namespace clang {
+namespace refactor {
+
+// FIXME: Implement two following classes.
+// SymbolDeclaration and SymbolUsage might reuse clang::tooling::Range.
+struct SymbolDeclaration {};
+struct SymbolUsage {};
+
+class SymbolIndexClient {
+public:
+  std::vector<SymbolDeclaration> getDeclarationsForUSR(StringRef USR);
+  std::vector<SymbolUsage> getReferencesForUSR(StringRef USR);
+};
+
+} // end namespace refactor
+} // end namespace refactor
+
+#endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_REFACTOR_SYMBOL_INDEX_CLIENT_H

clang-refactor/driver/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Support
+  )
+
+add_clang_executable(clang-refactor
+  Driver.cpp
+  )
+
+target_link_libraries(clang-refactor
+  clangAST
+  clangASTMatchers
+  clangBasic
+  clangFrontend
+  clangTooling
+
+  clangRefactorCore
+  )

clang-refactor/driver/Driver.cpp

+//===--- Driver.cpp - clang-refactor ----------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+///  \file This file implements a clang-refactor tool.
+///
+////===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include "ModuleManager.h"
+
+namespace clang {
+namespace refactor {
+
+static int printHelpMessage(ModuleManager &Manager) {
+  llvm::outs() << Manager.getTopLevelHelp();
+  return 1;
+}
+
+int clangRefactorMain(int argc, const char **argv) {
+  ModuleManager Manager;
+  if (argc > 1) {
+    llvm::StringRef FirstArgument(argv[1]);
+    if (FirstArgument == "-h" || FirstArgument == "-help" ||
+        FirstArgument == "--help")
+      return printHelpMessage(Manager);
+    const std::string Command = argv[1];
+    argv[1] = (llvm::Twine(argv[0]) + " " + llvm::Twine(argv[1])).str().c_str();
+    Manager.dispatch(Command, argc - 1, argv + 1);
+  } else {
+    return printHelpMessage(Manager);
+  }
+  return 0;
+}
+
+} // end namespace refactor
+} // end namespace clang
+
+int main(int argc, const char **argv) {
+  return clang::refactor::clangRefactorMain(argc, argv);
+}