[Tooling] A new framework for executing clang frontend actions.

[ioeric]

Summary

This defines a clang::tooling::ToolExecutor interface that can be extended to support different execution plans including standalone execution on a given set of TUs or parallel execution on all TUs in a codebase.

In order to enable multiprocessing execution, tool actions are expected to output result into a ToolResults interface provided by executors. The ToolResults interface abstracts how results are stored e.g. in-memory for standalone executions or on-disk for large-scale execution.

New executors can be registered as ToolExecutorPlugins via the ToolExecutorPluginRegistry. CLI tools can use createExecutorFromCommandLineArgs to create a specific registered executor according to the command-line arguments.

This patch also implements StandaloneToolExecutor which has the same behavior as the current ClangTool interface, i.e. execute frontend actions on a given set of TUs. At this point, it's simply a wrapper around ClangTool at this point.

This is still experimental but expected to replace the existing ClangTool interface so that specific tools would not need to worry about execution.

[ioeric]

• Merged with origin/master

[ioeric]

• Added ToolResults interface and moved classes to right files.

[ioeric]

• Minor cleanups.

[hokein]

This is a good start, a few comments. Also I'd suggest running clang-format on this patch -- I saw a few places violate the code style.

include/clang/Tooling/CommonOptionsParser.h

89–90

... is set to true for clarify.

112

return llvm::StringRef?

include/clang/Tooling/Execution.h

22

Consider moving this comment to ToolExecutor class to make it more discoverable?

48

How about using std::pair<std::string, std::string> type here? It would make it consistent with the allKVResults below.

Also I think we can define an alias KVResult for this type.

49

nit: all

82

nit: execute

lib/Tooling/Execution.cpp

62

Consider we have two plugins, the first iteration fails, and the second one succeeds, the code looks like still treating it as an error here? And if the case where there are more than one executor being created successfully, we just return the first one.

My understanding of the createExecutorFromCommandLineArgs's behavior is to find a proper ToolExecutorPlugin and create a ToolExecutor instance.

Maybe add a command-line flag like --executor=<my_executor_name> to make it straightforward to choose which registered executor is used, and make StandaloneToolExecutor as default?

74

nit: Executor

unittests/Tooling/ToolingTest.cpp

32 ↗

(On Diff #118898)

Seems that it is unused.

638 ↗

(On Diff #118898)

any reason to make it static?

667 ↗

(On Diff #118898)

nit: size_t, the same below.

[ioeric]

• clang-format code.

[ioeric]

• Address review comments.

[ioeric]

Thanks for the review!

include/clang/Tooling/CommonOptionsParser.h

112

I'm not a big fan of StringRef as a return value. Callers can simply use StringRef s = parser.getErrorMessage(), if they want to; otherwise, they would need to worry about the live time of s in auto s = parser.getErrorMessage(), if the interface returns a StringRef.

include/clang/Tooling/Execution.h

48

It would make the the interface a bit cumbersome to use - callers would need to make pairs all the time. And converting to pairs is an unnecessary performance overhead IMO. The consistency with AllKVResults is not that important after all.

We don't have a lot usages of std::pair<std::string, std::string> yet. I'll add an alias when needed. Currently, I think it makes sense to keep the result type explicit.

49

I think for-each is a more canonical name? E.g. we have for_each loop in C++.

lib/Tooling/Execution.cpp

62

Yes, we simply return the first plugin that matches.

It's hard to ensure that all executors have mutually exclusive arguments when more executors are added; it should be up to the executors to define unique (or as-unique-as-possible) arguments. For example, a map-reduce-based executor could require a "--mr" option to be set. The factory should simply return the first one that matches.

The point of this helper function is to hide the dispatching of different executors, so I don't think requiring users to specify the executor name is sensible. For example, users should only care about whether to set --mr instead of having to know the existence of different executors.

unittests/Tooling/ToolingTest.cpp

638 ↗

(On Diff #118898)

Added a comment explaining this.

667 ↗

(On Diff #118898)

createExecutorFromCommandLineArgs and, more importantly, the underlying library functions take int, so... :(

[ioeric]

• Add a ExecutionContext class.

[arphaman]

include/clang/Tooling/CommonOptionsParser.h

114

Would it be better to have an `llvm::Error' instead of the two fields?

include/clang/Tooling/Execution.h

48

You don't need to use the llvm:: prefix for StringRef.

77

Why not unique_ptr/shared_ptr? Who owns the results?

135

Standalone is one word.

137

Maybe this class and InMemoryToolResults should be in a separate header?

[ioeric]

• Fix broken unit tests when they are run in threads in the same process.

[ioeric]

• Move StandaloneToolExecutor into a separate header; added ExecutionTest.cpp.

[ioeric]

• Addressed review comments.

[ioeric]

Thanks for the review!

include/clang/Tooling/CommonOptionsParser.h

114

It would make the code a bit more complicated... we would need to convert between Error and strings a few times here and in the user code.

include/clang/Tooling/Execution.h

77

The context creator owns the results. Here we only store a reference.

137

Moved StandaloneToolExecutor into a separate header. I am keeping InMemoryToolResults here since it might be used by other executors.

[ioeric]

• Minor cleanup in tests.

[klimek]

include/clang/Tooling/CommonOptionsParser.h

89–90

If we want error handling, why not make it a static factory that returns an ErrorOr instead?

include/clang/Tooling/Execution.h

27–28

I'd remove this line. Those tend to only get stale without anybody changing them :)

50

I think = default is the new cool way.

52–54

Why do we need to get the results via the interface? For example, in a map/reduce style setup getting the results is infeasible.

77

I think it's weird that this is virtual, but we also have a default implementation that returns something that we require in the constructor. Either make that virtual, and don't put in a default implementation, or make it non-virtual.

91

I don't think we need to annotate every unowned pointer. Pointers are unowned by default, otherwise we'd use unique_ptr.

165–167

I think for clang-refactor we'll need one level of abstraction in the middle:
We'll need to be able to say "run on all translation units referencing symbol X" - how will this fit into the design?
Also, how will code changes generally fit with this design?

[ioeric]

• Add a factory method for creating CommonOptionsParser.
• Minor cleanup in CommonOptionsParser.
• Revert CommonOptionsParser changes.
• Merge branch 'option' into arcpatch-D34272

[ioeric]

• Address review comments.

include/clang/Tooling/CommonOptionsParser.h

89–90

Done. I pulled the changes for CommonOptionsParser into a separate patch (D39042 ). PTAL

include/clang/Tooling/Execution.h

27–28

Right :)

52–54

This would enable tools to access results regardless of the underlying representation of the results.

In a map/reduce style execution, we could have an implementation that deals with fetching and reading remote files; information about remote files can be fed into the implementation from the executor that performs such execution. It might make sense to add an interface that returns the metadata about the underlying data though (e.g. URI for remote files). WDYT?

165–167

I think for clang-refactor we'll need one level of abstraction in the middle:
We'll need to be able to say "run on all translation units referencing symbol X" - how will this fit into the design?

Yes. Firstly, I think clang-refactor is responsible for getting all TUs that reference a symbols and feeding the set of TUs into the tool interface. And then we could implements a "smart" executor that picks an appropriate execution (e.g. delegate to standalone executor or map/reduce style execution) according to the number of TUs.

Also, how will code changes generally fit with this design?

Code changes will be reported by tools/actions via the ToolResults interface as key-value pairs (e.g. source location + serialized AtomicChange). After the execution, callers would have access to the results in ToolResults, or we could also provide an interface that returns the information about the results so that they can be accessed later.

[arphaman]

include/clang/Tooling/StandaloneExecution.h

2

StandaloneExecution.h?

[ioeric]

• Remove unused variable.
• Narrow supported actions to FrontendActionFactory from ToolAction.

[ioeric]

• Experimented and refined the interfaces. o Get rid of ExecutionConfig class; pass actions into executors directly. o Add ArgumentAdjuster to ExecutionContext. o Make command-line based executor selection more explicit by requiring a "--executor" option.

[ioeric]

PTAL

I experimented with our internal tools (incl. mapreduce-based execution) and modified the interfaces a bit:
o Get rid of ExecutionConfig class; pass actions into executors directly.
o Moved ArgumentAdjusters that are common to all actions into ExecutionContext.
o Make command-line based executor selection more explicit by requiring an --executor option.

[ioeric]

• Get rid of the unused 'createExecutorByName' interface.

[ioeric]

• Only expose result reporting interface from ExecutionContext so that callbacks don't have access to results.

[ioeric]

• Minor comment update.

[ioeric]

include/clang/Tooling/Execution.h

52–54

After offline discussion, I now understand the concern about having tool callbacks be able to access results.

I've changed the ExecutionContext to only expose a result reporting interface instead of returning a reference of ToolResults - callbacks are expected to only have access to ExecutionContext. The ToolResults would only be available from the executor itself.

[klimek]

include/clang/Tooling/Execution.h

77

I think the argument adjuster adjustment shouldn't be part of this interface, as the argument adjusters cannot be changed in the phase in which we want the ExecutionContext to be used.

I'd just make the argument adjusters a parameter on the constructor here (or alternatively, do not couple them in here, and just hand them around as a separate entity).

131–132

I'd put the ArgumentAdjust second (as the action is the primary thing being acted on).

[ioeric]

• Addressed review comments o Removed argument adjusting interfaces from ExecutionContext o Switched positions of Action and Adjuster parameters in the execute interfaces

[ioeric]

include/clang/Tooling/Execution.h

77

You are right. After a second thought, the adjuster here is actually only used by the executor itself (e.g. default args), and executors can maintain their own adjusters internally, so it doesn't need to be in the interface.

I've removed the adjuster interfaces from ExecutionContext. Action-specific adjusters can still be passed in through the ToolExecutor::execute(...) interface.

[klimek]

lg

[FlameTop]

Hello, I'm afraid this commit is failing on Windows.

http://lab.llvm.org:8011/console?builder=llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast&builder=llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast

FAIL: Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleActionWithResult (14523 of 35936)

• TEST 'Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleActionWithResult' FAILED ****

Note: Google Test filter = StandaloneToolTest.SimpleActionWithResult

[==========] Running 1 test from 1 test case.

[----------] Global test environment set-up.

[----------] 1 test from StandaloneToolTest

[ RUN ] StandaloneToolTest.SimpleActionWithResult

C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.src\tools\clang\unittests\Tooling\ExecutionTest.cpp(217): error: Value of: !Err

Actual: false

Expected: true

error: error reading 'C:\a.cc'

1 error generated.

Error while processing C:\a.cc.

Program aborted due to an unhandled Error:

Failed to run action.LLVMSymbolizer: error reading file: PDB Error: Unable to load PDB. Make sure the file exists and is readable. Calling loadDataForExe

#0 0x0114e867 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x45e867)

#1 0x76805422 (C:\WINDOWS\System32\ucrtbase.dll+0xa5422)

#2 0x00d67dec (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x77dec)

#3 0x011820a4 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x4920a4)

---

PASS: Extra Tools Unit Tests :: change-namespace/./ChangeNamespaceTests.exe/ChangeNamespaceTest.SimpleMoveEnum (14524 of 35936)
PASS: Extra Tools Unit Tests :: change-namespace/./ChangeNamespaceTests.exe/ChangeNamespaceTest.DontMoveForwardDeclarationInClass (14525 of 35936)
FAIL: Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleAction (14526 of 35936)

• TEST 'Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleAction' FAILED ****

Note: Google Test filter = StandaloneToolTest.SimpleAction

[==========] Running 1 test from 1 test case.

[----------] Global test environment set-up.

[----------] 1 test from StandaloneToolTest

[ RUN ] StandaloneToolTest.SimpleAction

C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.src\tools\clang\unittests\Tooling\ExecutionTest.cpp(204): error: Value of: !Err

Actual: false

Expected: true

error: error reading 'C:\a.cc'

1 error generated.

Error while processing C:\a.cc.

Program aborted due to an unhandled Error:

Failed to run action.LLVMSymbolizer: error reading file: PDB Error: Unable to load PDB. Make sure the file exists and is readable. Calling loadDataForExe

#0 0x0114e867 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x45e867)

#1 0x76805422 (C:\WINDOWS\System32\ucrtbase.dll+0xa5422)

#2 0x00d67dec (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x77dec)

#3 0x011820a4 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x4920a4)

---

[bjope]

I get errors when compiling due to this commit:

FAILED: tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o 
/repo/app/clang/3.6/bin/clang++  -march=corei7  -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/lib/Tooling -I../tools/clang/lib/Tooling -I../tools/clang/include -Itools/clang/include -Iinclude -I../include -I/repo/app/valgrind/3.11.0/include  -fPIC -fvisibility-inlines-hidden -Werror -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fcolor-diagnostics -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3    -UNDEBUG  -fno-exceptions -fno-rtti -MMD -MT tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o -MF tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o.d -o tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o -c ../tools/clang/lib/Tooling/StandaloneExecution.cpp
../tools/clang/lib/Tooling/StandaloneExecution.cpp:65:11: error: unused variable 'Ret' [-Werror,-Wunused-variable]
  if (int Ret = Tool.run(Action.first.get()))
          ^
1 error generated.

include/clang/Tooling/CommonOptionsParser.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLING_COMMONOPTIONSPARSER_H
 #define LLVM_CLANG_TOOLING_COMMONOPTIONSPARSER_H
 
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Error.h"
 
 namespace clang {
 namespace tooling {
 /// \brief A parser for options common to all command-line Clang tools.
 ///
 /// Parses a common subset of command-line arguments, locates and loads a
 /// compilation commands database and runs a tool with user-specified action. It
 /// also contains a help message for the common command-line options.
public:
 
   /// \brief Parses command-line, initializes a compilation database.
   ///
   /// This constructor can change argc and argv contents, e.g. consume
   /// command-line options used for creating FixedCompilationDatabase.
   ///
   /// All options not belonging to \p Category become hidden.
   ///
   /// It also allows calls to set the required number of positional parameters.
-  ///
-  /// This constructor exits program in case of error.
   CommonOptionsParser(int &argc, const char **argv,
                       llvm::cl::OptionCategory &Category,
                       llvm::cl::NumOccurrencesFlag OccurrencesFlag,
                       const char *Overview = nullptr);
 
+  static llvm::Expected<std::unique_ptr<CommonOptionsParser>>
+  create(int &argc, const char **argv, llvm::cl::OptionCategory &Category,
+         llvm::cl::NumOccurrencesFlag OccurrencesFlag,
+         const char *Overview = nullptr);
+
   /// Returns a reference to the loaded compilations database.
   CompilationDatabase &getCompilations() {
     return *Compilations;
   }
 
   /// Returns a list of source file paths to process.
   const std::vector<std::string> &getSourcePathList() const {
     return SourcePathList;
   }
 
   static const char *const HelpMessage;
 
 private:
+  CommonOptionsParser() = default;
+
+  static llvm::Error init(int &argc, const char **argv,
+                          llvm::cl::OptionCategory &Category,
+                          llvm::cl::NumOccurrencesFlag OccurrencesFlag,
+                          const char *Overview, CommonOptionsParser *Parser);
+
   std::unique_ptr<CompilationDatabase> Compilations;
   std::vector<std::string> SourcePathList;
 };
 
 class ArgumentsAdjustingCompilations : public CompilationDatabase {
 public:
   ArgumentsAdjustingCompilations(
       std::unique_ptr<CompilationDatabase> Compilations)

include/clang/Tooling/Execution.h

+//===--- Execution.h - Executing clang frontend actions -*- C++ ---------*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file defines framework for executing clang frontend actions.
+//
+//  The framework can be extended to support different execution plans including
+//  standalone execution on the given TUs or parallel execution on all TUs in
+//  the codebase.
+//
+//  In order to enable multiprocessing execution, tool actions are expected to
+//  output result into the ToolResults provided by the executor. The
+//  `ToolResults` is an interface that abstracts how results are stored e.g.
+//  in-memory for standalone execution or on-disk for large-scale execution.
+//
+//  New executors can be registered as ToolExecutorPlugins via the
+//  `ToolExecutorPluginRegistry`. CLI tools can use
+//  `createExecutorFromCommandLineArgs` to create a specific registered executor
+//  according to the command-line arguments.
+//
+//  This is still experimental but expected to replace the existing `ClangTool`
+//  interface.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_EXECUTION_H
+#define LLVM_CLANG_TOOLING_EXECUTION_H
+
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Registry.h"
+
+namespace clang {
+namespace tooling {
+
+/// \brief An abstraction for the result of a tool execution. For example, the
+/// underlying result can be in-memory or on-disk.
+///
+/// Results should be string key-value pairs. For example, a refactoring tool
+/// can use source location as key and a replacement in YAML format as value.
+class ToolResults {
+public:
+  virtual ~ToolResults() {}
+  virtual void addResult(StringRef Key, StringRef Value) = 0;
+  virtual std::vector<std::pair<std::string, std::string>> AllKVResults() = 0;
+  virtual void forEachResult(
+      llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) = 0;
+};
+
+class InMemoryToolResults : public ToolResults {
+public:
+  void addResult(StringRef Key, StringRef Value) override;
+  std::vector<std::pair<std::string, std::string>> AllKVResults() override;
+  void forEachResult(llvm::function_ref<void(StringRef Key, StringRef Value)>
+                         Callback) override;
+
+private:
+  std::vector<std::pair<std::string, std::string>> KVResults;
+};
+
+/// \brief The context of an execution, including the information about
+/// compilation and results.
+class ExecutionContext {
+public:
+  virtual ~ExecutionContext() {}
+
+  /// \brief Initializes a context. This does not take ownership of `Results`.
+  explicit ExecutionContext(ToolResults *Results) : Results(Results) {}
+
+  virtual ToolResults *getToolResults() const { return Results; }
+
+  // Returns the source control system's revision number if applicable.
+  // Otherwise returns an empty string.
+  virtual std::string getRevision() { return ""; }
+
+  // Returns the corpus being analyzed, e.g. "llvm" for the LLVM codebase, if
+  // applicable.
+  virtual std::string getCorpus() { return ""; }
+
+  // Returns the currently processed compilation unit if available.
+  virtual std::string getCurrentCompilationUnit() { return ""; }
+
+private:
+  // A reference to the results container. Not owned!
+  ToolResults *Results;
+};
+
+/// \brief Specifies the actions to be executed in a tool execution and the
+/// compilation arguments to be used.
+struct ExecutionConfig {
+  ExecutionConfig() = default;
+
+  // Config for executing a single `Action` with the default configuration e.g.
+  // `ClangStripOutputAdjuster` and `ClangSyntaxOnlyAdjuster`.
+  explicit ExecutionConfig(std::unique_ptr<ToolAction> Action);
+
+  // Config for executing multiple actions that use customized argument
+  // adjusters in addition to the default one.
+  std::vector<std::pair<ArgumentsAdjuster, std::unique_ptr<ToolAction>>>
+      Actions;
+};
+
+/// \brief Interface for executing clang frontend actions.
+///
+/// This can be extended to support running tool actions in different
+/// execution mode, e.g. on a specific set of TUs or many TUs in parallel.
+///
+///  New executors can be registered as ToolExecutorPlugins via the
+///  `ToolExecutorPluginRegistry`. CLI tools can use
+///  `createExecutorFromCommandLineArgs` to create a specific registered
+///  executor according to the command-line arguments.
+class ToolExecutor {
+public:
+  virtual ~ToolExecutor() {}
+
+  /// \brief Returns the name of a specific executor.
+  virtual StringRef getExecutorName() const = 0;
+
+  /// \brief Executes actions specified in the execution configuration.
+  virtual llvm::Error execute(const ExecutionConfig &Config) = 0;
+
+  /// \brief Returns a reference of the execution context.
+  virtual ExecutionContext *getExecutionContext() = 0;
+
+  /// \brief Map a virtual file to be used while running the tool.
+  ///
+  /// \param FilePath The path at which the content will be mapped.
+  /// \param Content A buffer of the file's content.
+  virtual void mapVirtualFile(StringRef FilePath, StringRef Content) = 0;
+};
+
+/// \brief Interface for factories that create speicifc executors based on the
+/// commandline arguments. This is also used as a plugin to be registered into
+/// ToolExecutorPluginRegistry.
+class ToolExecutorPlugin {
+public:
+  virtual ~ToolExecutorPlugin() {}
+
+  /// \brief Create a ToolExecutor based on the commandline arguments. If the
+  /// set of commandline arguments is not supported by the ToolExecutor, this
+  /// returns an error.
+  /// Expected commandline options should be registered in the implementation.
+  virtual llvm::Expected<std::unique_ptr<ToolExecutor>>
+  create(int &argc, const char **argv, llvm::cl::OptionCategory &Category) = 0;
+};
+
+/// \brief This creates a ToolExecutor that is in the global registry and
+/// supports the given commandline arguments.
+///
+/// This tries creating all registered executors from the given command-line
+/// arguments and returns the first executor that matches all arguments.
+///
+/// This should only be called once in a program.
+///
+/// By default, this creates a `StandaloneToolExecutor` if not other tool
+/// flags are used.
+llvm::Expected<std::unique_ptr<ToolExecutor>>
+createExecutorFromCommandLineArgs(int &argc, const char **argv,
+                                  llvm::cl::OptionCategory &Category,
+                                  const char *Overview = nullptr);
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_EXECUTION_H

include/clang/Tooling/StandaloneExecution.h

+//===--- Execution.h - Standalone clang action execution. -*- C++ -------*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file defines standalone execution of clang tools.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
+#define LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
+
+#include "clang/Tooling/Execution.h"
+
+namespace clang {
+namespace tooling {
+
+/// \brief A standalone executor that runs FrontendActions on a given set of
+/// TUs in sequence.
+class StandaloneToolExecutor : public ToolExecutor {
+public:
+  static const char *ExecutorName;
+
+  /// \brief Init with \p CompilationDatabase and the paths of all files to be
+  /// proccessed.
+  StandaloneToolExecutor(
+      const CompilationDatabase &Compilations,
+      llvm::ArrayRef<std::string> SourcePaths,
+      std::shared_ptr<PCHContainerOperations> PCHContainerOps =
+          std::make_shared<PCHContainerOperations>());
+
+  /// \brief Init with \p CommonOptionsParser. This is expected to be used by
+  /// `createExecutorFromCommandLineArgs` based on commandline options.
+  ///
+  /// The executor takes ownership of \p Options.
+  StandaloneToolExecutor(
+      std::unique_ptr<CommonOptionsParser> Options,
+      std::shared_ptr<PCHContainerOperations> PCHContainerOps =
+          std::make_shared<PCHContainerOperations>());
+
+  StringRef getExecutorName() const override { return ExecutorName; }
+
+  llvm::Error execute(const ExecutionConfig &Config) override;
+
+  /// \brief Set a \c DiagnosticConsumer to use during parsing.
+  void setDiagnosticConsumer(DiagnosticConsumer *DiagConsumer) {
+    Tool.setDiagnosticConsumer(DiagConsumer);
+  }
+
+  /// \brief Returns a reference to the execution context. Tool results will be
+  /// put into the results container in the context during the execution.
+  ExecutionContext *getExecutionContext() override { return &Context; };
+
+  llvm::ArrayRef<std::string> getSourcePaths() const {
+    return Tool.getSourcePaths();
+  }
+
+  void mapVirtualFile(StringRef FilePath, StringRef Content) override {
+    Tool.mapVirtualFile(FilePath, Content);
+  }
+
+  /// \brief Returns the file manager used in the tool.
+  ///
+  /// The file manager is shared between all translation units.
+  FileManager &getFiles() { return Tool.getFiles(); }
+
+private:
+  // Used to store the parser when the executor is initialized with parser.
+  std::unique_ptr<CommonOptionsParser> OptionsParser;
+  // FIXME: The standalone executor is currently just a wrapper of `ClangTool`.
+  // Merge `ClangTool` implementation into the this.
+  ClangTool Tool;
+  ExecutionContext Context;
+  InMemoryToolResults Results;
+};
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H

include/clang/Tooling/ToolExecutorPluginRegistry.h

+//===--- ToolExecutorPluginRegistry.h - -------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
+#define LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
+
+#include "clang/Tooling/Execution.h"
+#include "llvm/Support/Registry.h"
+
+namespace clang {
+namespace tooling {
+
+typedef llvm::Registry<ToolExecutorPlugin> ToolExecutorPluginRegistry;
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H

include/clang/Tooling/Tooling.h

 //  - running match/replace tools over C++ code
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLING_TOOLING_H
 #define LLVM_CLANG_TOOLING_TOOLING_H
 
 #include "clang/AST/ASTConsumer.h"
-#include "clang/Frontend/PCHContainerOperations.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/LLVM.h"
 #include "clang/Driver/Util.h"
 #include "clang/Frontend/FrontendAction.h"
+#include "clang/Frontend/PCHContainerOperations.h"
 #include "clang/Lex/ModuleLoader.h"
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Option/Option.h"
 #include <memory>
 #include <string>
public:
   /// append them to ASTs.
   int buildASTs(std::vector<std::unique_ptr<ASTUnit>> &ASTs);
 
   /// \brief Returns the file manager used in the tool.
   ///
   /// The file manager is shared between all translation units.
   FileManager &getFiles() { return *Files; }
 
- private:
+  llvm::ArrayRef<std::string> getSourcePaths() const { return SourcePaths; }
+
   const CompilationDatabase &Compilations;
   std::vector<std::string> SourcePaths;
   std::shared_ptr<PCHContainerOperations> PCHContainerOps;
 
   llvm::IntrusiveRefCntPtr<vfs::OverlayFileSystem> OverlayFileSystem;
   llvm::IntrusiveRefCntPtr<vfs::InMemoryFileSystem> InMemoryFileSystem;
   llvm::IntrusiveRefCntPtr<FileManager> Files;
   // Contains a list of pairs (<file name>, <file content>).

lib/Tooling/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Option
   Support
   )
 
 add_subdirectory(Core)
 add_subdirectory(Refactoring)
 add_subdirectory(ASTDiff)
 
 add_clang_library(clangTooling
   ArgumentsAdjusters.cpp
   CommonOptionsParser.cpp
   CompilationDatabase.cpp
+  Execution.cpp
   FileMatchTrie.cpp
   FixIt.cpp
   JSONCompilationDatabase.cpp
   Refactoring.cpp
   RefactoringCallbacks.cpp
+  StandaloneExecution.cpp
   Tooling.cpp
 
   DEPENDS
   ClangDriverOptions
 
   LINK_LIBS
   clangAST
   clangASTMatchers
   clangBasic
   clangDriver
   clangFormat
   clangFrontend
   clangLex
   clangRewrite
   clangToolingCore
   )

lib/Tooling/CommonOptionsParser.cpp

 //  It creates a CompilationDatabase and reads common command-line options.
 //
 //  This class uses the Clang Tooling infrastructure, see
 //    http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html
 //  for details on setting it up with LLVM source tree.
 //
 //===----------------------------------------------------------------------===//
 
-#include "llvm/Support/CommandLine.h"
 #include "clang/Tooling/CommonOptionsParser.h"
 #include "clang/Tooling/Tooling.h"
+#include "llvm/Support/CommandLine.h"
 
 using namespace clang::tooling;
 using namespace llvm;
 
 const char *const CommonOptionsParser::HelpMessage =
     "\n"
     "-p <build-path> is used to read a compile command database.\n"
     "\n"

 std::vector<CompileCommand> ArgumentsAdjustingCompilations::adjustCommands(
     std::vector<CompileCommand> Commands) const {
   for (CompileCommand &Command : Commands)
     for (const auto &Adjuster : Adjusters)
       Command.CommandLine = Adjuster(Command.CommandLine, Command.Filename);
   return Commands;
 }
 
-CommonOptionsParser::CommonOptionsParser(
-    int &argc, const char **argv, cl::OptionCategory &Category,
-    llvm::cl::NumOccurrencesFlag OccurrencesFlag, const char *Overview) {
+llvm::Error
+CommonOptionsParser::init(int &argc, const char **argv,
+                          cl::OptionCategory &Category,
+                          llvm::cl::NumOccurrencesFlag OccurrencesFlag,
+                          const char *Overview, CommonOptionsParser *Parser) {
   static cl::opt<bool> Help("h", cl::desc("Alias for -help"), cl::Hidden,
                             cl::sub(*cl::AllSubCommands));
 
   static cl::opt<std::string> BuildPath("p", cl::desc("Build path"),
                                         cl::Optional, cl::cat(Category),
                                         cl::sub(*cl::AllSubCommands));
 
   static cl::list<std::string> SourcePaths(
       cl::Positional, cl::desc("<source0> [... <sourceN>]"), OccurrencesFlag,
       cl::cat(Category), cl::sub(*cl::AllSubCommands));
 
   static cl::list<std::string> ArgsAfter(
       "extra-arg",
       cl::desc("Additional argument to append to the compiler command line"),
       cl::cat(Category), cl::sub(*cl::AllSubCommands));
 
   static cl::list<std::string> ArgsBefore(
       "extra-arg-before",
       cl::desc("Additional argument to prepend to the compiler command line"),
       cl::cat(Category), cl::sub(*cl::AllSubCommands));
 
+  cl::ResetAllOptionOccurrences();
+
   cl::HideUnrelatedOptions(Category);
 
   std::string ErrorMessage;
-  Compilations =
+  Parser->Compilations =
       FixedCompilationDatabase::loadFromCommandLine(argc, argv, ErrorMessage);
-  if (!Compilations && !ErrorMessage.empty())
-    llvm::errs() << ErrorMessage;
-  cl::ParseCommandLineOptions(argc, argv, Overview);
+  if (!ErrorMessage.empty())
+    ErrorMessage.append("\n");
+  llvm::raw_string_ostream OS(ErrorMessage);
+  // Stop initializing if command-line option parsing failed.
+  if (!cl::ParseCommandLineOptions(argc, argv, Overview, &OS)) {
+    OS.flush();
+    return llvm::make_error<llvm::StringError>("[CommonOptionsParser]: " +
+                                                   ErrorMessage,
+                                               llvm::inconvertibleErrorCode());
+  }
+
   cl::PrintOptionValues();
 
-  SourcePathList = SourcePaths;
+  Parser->SourcePathList = SourcePaths;
   if ((OccurrencesFlag == cl::ZeroOrMore || OccurrencesFlag == cl::Optional) &&
-      SourcePathList.empty())
-    return;
-  if (!Compilations) {
+      Parser->SourcePathList.empty())
+    return llvm::Error::success();
+  if (!Parser->Compilations) {
     if (!BuildPath.empty()) {
-      Compilations =
+      Parser->Compilations =
           CompilationDatabase::autoDetectFromDirectory(BuildPath, ErrorMessage);
     } else {
-      Compilations = CompilationDatabase::autoDetectFromSource(SourcePaths[0],
-                                                               ErrorMessage);
+      Parser->Compilations = CompilationDatabase::autoDetectFromSource(
+          SourcePaths[0], ErrorMessage);
     }
-    if (!Compilations) {
+    if (!Parser->Compilations) {
       llvm::errs() << "Error while trying to load a compilation database:\n"
                    << ErrorMessage << "Running without flags.\n";
-      Compilations.reset(
+      Parser->Compilations.reset(
           new FixedCompilationDatabase(".", std::vector<std::string>()));
     }
   }
   auto AdjustingCompilations =
       llvm::make_unique<ArgumentsAdjustingCompilations>(
-          std::move(Compilations));
+          std::move(Parser->Compilations));
   AdjustingCompilations->appendArgumentsAdjuster(
       getInsertArgumentAdjuster(ArgsBefore, ArgumentInsertPosition::BEGIN));
   AdjustingCompilations->appendArgumentsAdjuster(
       getInsertArgumentAdjuster(ArgsAfter, ArgumentInsertPosition::END));
-  Compilations = std::move(AdjustingCompilations);
+  Parser->Compilations = std::move(AdjustingCompilations);
+  return llvm::Error::success();
+}
+
+llvm::Expected<std::unique_ptr<CommonOptionsParser>>
+CommonOptionsParser::create(int &argc, const char **argv,
+                            llvm::cl::OptionCategory &Category,
+                            llvm::cl::NumOccurrencesFlag OccurrencesFlag,
+                            const char *Overview) {
+  std::unique_ptr<CommonOptionsParser> Parser(new CommonOptionsParser);
+  llvm::Error Err =
+      init(argc, argv, Category, OccurrencesFlag, Overview, Parser.get());
+  if (Err)
+    return std::move(Err);
+  return Parser;
+}
+
+CommonOptionsParser::CommonOptionsParser(
+    int &argc, const char **argv, cl::OptionCategory &Category,
+    llvm::cl::NumOccurrencesFlag OccurrencesFlag, const char *Overview) {
+  llvm::Error Err = init(argc, argv, Category, OccurrencesFlag, Overview, this);
+  if (Err) {
+    llvm::report_fatal_error(
+        "CommonOptionsParser: failed to parse command-line arguments. " +
+        llvm::toString(std::move(Err)));
+  }
 }

lib/Tooling/Execution.cpp

+//===- lib/Tooling/Execution.cpp - Implements tool execution framework. ---===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/ToolExecutorPluginRegistry.h"
+#include "clang/Tooling/Tooling.h"
+
+LLVM_INSTANTIATE_REGISTRY(clang::tooling::ToolExecutorPluginRegistry)
+
+namespace clang {
+namespace tooling {
+
+void InMemoryToolResults::addResult(StringRef Key, StringRef Value) {
+  KVResults.push_back({Key.str(), Value.str()});
+}
+
+std::vector<std::pair<std::string, std::string>>
+InMemoryToolResults::AllKVResults() {
+  return KVResults;
+}
+
+void InMemoryToolResults::forEachResult(
+    llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) {
+  for (const auto &KV : KVResults) {
+    Callback(KV.first, KV.second);
+  }
+}
+
+ExecutionConfig::ExecutionConfig(std::unique_ptr<ToolAction> Action) {
+  Actions.push_back(std::make_pair(nullptr, std::move(Action)));
+}
+
+llvm::Expected<std::unique_ptr<ToolExecutor>>
+createExecutorFromCommandLineArgs(int &argc, const char **argv,
+                                  llvm::cl::OptionCategory &Category,
+                                  const char *Overview) {
+  std::string ErrorMessage;
+  llvm::raw_string_ostream OS(ErrorMessage);
+  // Find the first executor plugin that matches the given command-line
+  // arguments.
+  for (auto I = ToolExecutorPluginRegistry::begin(),
+            E = ToolExecutorPluginRegistry::end();
+       I != E; ++I) {
+    std::unique_ptr<ToolExecutorPlugin> Plugin(I->instantiate());
+    std::vector<std::string> args(argv, argv + argc);
+    int arg_num = argc;
+    llvm::Expected<std::unique_ptr<ToolExecutor>> Executor =
+        Plugin->create(arg_num, argv, Category);
+    if (!Executor) {
+      OS << "Failed to create '" << I->getName()
+         << "': " << llvm::toString(Executor.takeError()) << "\n";
+      llvm::cl::ResetAllOptionOccurrences();
+      continue;
+    }
+    argc = arg_num;
+    return std::move(*Executor);
+  }
+  OS.flush();
+  return llvm::make_error<llvm::StringError>(ErrorMessage,
+                                             llvm::inconvertibleErrorCode());
+}
+
+} // end namespace tooling
+} // end namespace clang

lib/Tooling/StandaloneExecution.cpp

+//===- lib/Tooling/Execution.cpp - Standalone clang action execution. -----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/StandaloneExecution.h"
+#include "clang/Tooling/ToolExecutorPluginRegistry.h"
+
+namespace clang {
+namespace tooling {
+
+static llvm::Error make_string_error(const llvm::Twine &Message) {
+  return llvm::make_error<llvm::StringError>(Message,
+                                             llvm::inconvertibleErrorCode());
+}
+
+const char *StandaloneToolExecutor::ExecutorName = "StandaloneToolExecutor";
+
+StandaloneToolExecutor::StandaloneToolExecutor(
+    const CompilationDatabase &Compilations,
+    llvm::ArrayRef<std::string> SourcePaths,
+    std::shared_ptr<PCHContainerOperations> PCHContainerOps)
+    : Tool(Compilations, SourcePaths), Context(&Results) {}
+
+StandaloneToolExecutor::StandaloneToolExecutor(
+    std::unique_ptr<CommonOptionsParser> Options,
+    std::shared_ptr<PCHContainerOperations> PCHContainerOps)
+    : OptionsParser(std::move(Options)),
+      Tool(OptionsParser->getCompilations(), OptionsParser->getSourcePathList(),
+           PCHContainerOps),
+      Context(&Results) {}
+
+llvm::Error StandaloneToolExecutor::execute(const ExecutionConfig &Config) {
+  if (Config.Actions.empty())
+    return make_string_error("No action to execute.");
+
+  if (Config.Actions.size() != 1)
+    return make_string_error(
+        "Only support executing exactly 1 ToolAction at this point.");
+
+  auto &Action = Config.Actions.front();
+  if (Action.first)
+    Tool.appendArgumentsAdjuster(Action.first);
+  if (int Ret = Tool.run(Action.second.get()))
+    return make_string_error("Failed to run ToolAction.");
+
+  return llvm::Error::success();
+}
+
+class StandaloneToolExecutorPlugin : public ToolExecutorPlugin {
+public:
+  llvm::Expected<std::unique_ptr<ToolExecutor>>
+  create(int &argc, const char **argv,
+         llvm::cl::OptionCategory &Category) override {
+    auto OptionsParser =
+        CommonOptionsParser::create(argc, argv, Category, llvm::cl::OneOrMore,
+                                    /*Overview=*/nullptr);
+    if (!OptionsParser)
+      return make_string_error("[StandaloneToolExecutorPlugin] " +
+                               llvm::toString(OptionsParser.takeError()));
+    if ((*OptionsParser)->getSourcePathList().empty())
+      return make_string_error(
+          "[StandaloneToolExecutorPlugin] No positional argument found.");
+    return llvm::make_unique<StandaloneToolExecutor>(std::move(*OptionsParser));
+  }
+};
+
+// This anchor is used to force the linker to link in the generated object file
+// and thus register the plugin.
+volatile int ToolExecutorPluginAnchorSource = 0;
+
+static ToolExecutorPluginRegistry::Add<StandaloneToolExecutorPlugin>
+    X("standalone-executor",
+      "Runs ToolActions on a set of files provided via positional arguments.");
+
+} // end namespace tooling
+} // end namespace clang

lib/Tooling/Tooling.cpp

 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/Option.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 #include <utility>
 
 #define DEBUG_TYPE "clang-tooling"

unittests/Tooling/CMakeLists.txt

 endif()
 
 add_clang_unittest(ToolingTests
   ASTSelectionTest.cpp
   CastExprTest.cpp
   CommentHandlerTest.cpp
   CompilationDatabaseTest.cpp
   DiagnosticsYamlTest.cpp
+  ExecutionTest.cpp
   FixItTest.cpp
   LexicallyOrderedRecursiveASTVisitorTest.cpp
   LookupTest.cpp
   QualTypeNamesTest.cpp
   RecursiveASTVisitorTest.cpp
   RecursiveASTVisitorTestCallVisitor.cpp
   RecursiveASTVisitorTestDeclVisitor.cpp
   RecursiveASTVisitorTestExprVisitor.cpp

unittests/Tooling/ExecutionTest.cpp

+//===- unittest/Tooling/ExecutionTest.cpp - Tool execution tests. --------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/AST/ASTConsumer.h"
+#include "clang/AST/DeclCXX.h"
+#include "clang/AST/RecursiveASTVisitor.h"
+#include "clang/Frontend/ASTUnit.h"
+#include "clang/Frontend/FrontendAction.h"
+#include "clang/Frontend/FrontendActions.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/StandaloneExecution.h"
+#include "clang/Tooling/ToolExecutorPluginRegistry.h"
+#include "clang/Tooling/Tooling.h"
+#include "gtest/gtest.h"
+#include <algorithm>
+#include <string>
+
+namespace clang {
+namespace tooling {
+
+namespace {
+
+// This traverses the AST and outputs function name as key and "1" as value for
+// each function declaration.
+class ASTConsumerWithResult
+    : public ASTConsumer,
+      public RecursiveASTVisitor<ASTConsumerWithResult> {
+public:
+  using ASTVisitor = RecursiveASTVisitor<ASTConsumerWithResult>;
+
+  explicit ASTConsumerWithResult(ExecutionContext *Context) : Context(Context) {
+    assert(Context != nullptr);
+  }
+
+  void HandleTranslationUnit(clang::ASTContext &Context) override {
+    TraverseDecl(Context.getTranslationUnitDecl());
+  }
+
+  bool TraverseFunctionDecl(clang::FunctionDecl *Decl) {
+    Context->getToolResults()->addResult(Decl->getNameAsString(), "1");
+    return ASTVisitor::TraverseFunctionDecl(Decl);
+  }
+
+private:
+  ExecutionContext *const Context;
+};
+
+class ReportResultAction : public ASTFrontendAction {
+public:
+  explicit ReportResultAction(ExecutionContext *Context) : Context(Context) {
+    assert(Context != nullptr);
+  }
+
+protected:
+  std::unique_ptr<clang::ASTConsumer>
+  CreateASTConsumer(clang::CompilerInstance &compiler,
+                    StringRef /* dummy */) override {
+    std::unique_ptr<clang::ASTConsumer> ast_consumer{
+        new ASTConsumerWithResult(Context)};
+    return ast_consumer;
+  }
+
+private:
+  ExecutionContext *const Context;
+};
+
+class ReportResultActionFactory : public FrontendActionFactory {
+public:
+  ReportResultActionFactory(ExecutionContext *Context) : Context(Context) {}
+  FrontendAction *create() override { return new ReportResultAction(Context); }
+
+private:
+  ExecutionContext *const Context;
+};
+
+inline llvm::Error make_string_error(const llvm::Twine &Message) {
+  return llvm::make_error<llvm::StringError>(Message,
+                                             llvm::inconvertibleErrorCode());
+}
+
+} // namespace
+
+class TestToolExecutor : public ToolExecutor {
+public:
+  static const char *ExecutorName;
+
+  TestToolExecutor(std::unique_ptr<CommonOptionsParser> Options)
+      : OptionsParser(std::move(Options)) {}
+
+  StringRef getExecutorName() const override { return ExecutorName; }
+
+  llvm::Error execute(const ExecutionConfig &) override {
+    return llvm::Error::success();
+  }
+
+  ExecutionContext *getExecutionContext() override { return nullptr; };
+
+  llvm::ArrayRef<std::string> getSourcePaths() const {
+    return OptionsParser->getSourcePathList();
+  }
+
+  void mapVirtualFile(StringRef FilePath, StringRef Content) override {
+    VFS[FilePath] = Content;
+  }
+
+private:
+  std::unique_ptr<CommonOptionsParser> OptionsParser;
+  std::string SourcePaths;
+  std::map<std::string, std::string> VFS;
+};
+
+const char *TestToolExecutor::ExecutorName = "TestToolExecutor";
+
+class TestToolExecutorPlugin : public ToolExecutorPlugin {
+public:
+  llvm::Expected<std::unique_ptr<ToolExecutor>>
+  create(int &argc, const char **argv,
+         llvm::cl::OptionCategory &Category) override {
+    // Depending on the test runner, test cases might run in different threads
+    // in the same process, so we put test options in the stack and manually
+    // remove it from the global registry later so that tests do not interfere
+    // with each other.
+    llvm::cl::opt<bool> TestExecutor("test_executor",
+                                     llvm::cl::desc("Use TestToolExecutor"));
+    auto OptionsParser = CommonOptionsParser::create(
+        argc, argv, Category, llvm::cl::OneOrMore, /*Overview=*/nullptr);
+    TestExecutor.removeArgument();
+    if (!OptionsParser)
+      return make_string_error("[TestToolExecutorPlugin] " +
+                               llvm::toString(OptionsParser.takeError()));
+    if (!TestExecutor)
+      return make_string_error(
+          "[TestToolExecutorPlugin] --test_executor is not set.");
+    return llvm::make_unique<TestToolExecutor>(std::move(*OptionsParser));
+  }
+};
+
+// This anchor is used to force the linker to link in the generated object file
+// and thus register the plugin.
+extern volatile int ToolExecutorPluginAnchorSource;
+
+static int LLVM_ATTRIBUTE_UNUSED TestToolExecutorPluginAnchorDest =
+    ToolExecutorPluginAnchorSource;
+
+static ToolExecutorPluginRegistry::Add<TestToolExecutorPlugin>
+    X("test-executor", "Plugin for TestToolExecutor.");
+
+llvm::cl::OptionCategory TestCategory("execution-test options");
+
+TEST(CreateToolExecutorTest, FailedCreateExecutorUndefinedFlag) {
+  std::vector<const char *> argv = {"prog", "--fake_flag_no_no_no", "f"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_FALSE((bool)Executor);
+  llvm::consumeError(Executor.takeError());
+}
+
+TEST(CreateToolExecutorTest, RegisterFlagsBeforeReset) {
+  llvm::cl::opt<std::string> BeforeReset(
+      "before_reset", llvm::cl::desc("Defined before reset."),
+      llvm::cl::init(""));
+
+  llvm::cl::ResetAllOptionOccurrences();
+
+  std::vector<const char *> argv = {"prog", "--before_reset=set", "f"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_TRUE((bool)Executor);
+  EXPECT_EQ(BeforeReset, "set");
+  BeforeReset.removeArgument();
+}
+
+TEST(CreateToolExecutorTest, CreateStandaloneToolExecutor) {
+  std::vector<const char *> argv = {"prog", "standalone.cpp"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_TRUE((bool)Executor);
+  EXPECT_EQ(Executor->get()->getExecutorName(),
+            StandaloneToolExecutor::ExecutorName);
+}
+
+TEST(CreateToolExecutorTest, CreateTestToolExecutor) {
+  std::vector<const char *> argv = {"prog", "test.cpp", "--test_executor"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_TRUE((bool)Executor);
+  EXPECT_EQ(Executor->get()->getExecutorName(), TestToolExecutor::ExecutorName);
+}
+
+TEST(StandaloneToolTest, SynctaxOnlyActionOnSimpleCode) {
+  FixedCompilationDatabase Compilations("/", std::vector<std::string>());
+  StandaloneToolExecutor Executor(Compilations,
+                                  std::vector<std::string>(1, "/a.cc"));
+  Executor.mapVirtualFile("/a.cc", "int x = 0;");
+
+  auto Err = Executor.execute(
+      ExecutionConfig(newFrontendActionFactory<SyntaxOnlyAction>()));
+  ASSERT_TRUE(!Err);
+}
+
+TEST(StandaloneToolTest, SimpleAction) {
+  FixedCompilationDatabase Compilations("/", std::vector<std::string>());
+  StandaloneToolExecutor Executor(Compilations,
+                                  std::vector<std::string>(1, "/a.cc"));
+  Executor.mapVirtualFile("/a.cc", "int x = 0;");
+
+  ExecutionContext *Context = Executor.getExecutionContext();
+
+  auto Err = Executor.execute(ExecutionConfig(
+      std::unique_ptr<ToolAction>(new ReportResultActionFactory(Context))));
+  ASSERT_TRUE(!Err);
+  auto KVs = Context->getToolResults()->AllKVResults();
+  ASSERT_EQ(KVs.size(), 0u);
+}
+
+TEST(StandaloneToolTest, SimpleActionWithResult) {
+  FixedCompilationDatabase Compilations("/", std::vector<std::string>());
+  StandaloneToolExecutor Executor(Compilations,
+                                  std::vector<std::string>(1, "/a.cc"));
+  Executor.mapVirtualFile("/a.cc", "int x = 0; void f() {}");
+
+  ExecutionContext *Context = Executor.getExecutionContext();
+
+  auto Err = Executor.execute(ExecutionConfig(
+      std::unique_ptr<ToolAction>(new ReportResultActionFactory(Context))));
+  ASSERT_TRUE(!Err);
+  auto KVs = Context->getToolResults()->AllKVResults();
+  ASSERT_EQ(KVs.size(), 1u);
+  EXPECT_EQ("f", KVs[0].first);
+  EXPECT_EQ("1", KVs[0].second);
+
+  Context->getToolResults()->forEachResult(
+      [](StringRef, StringRef Value) { EXPECT_EQ("1", Value); });
+}
+
+} // end namespace tooling
+} // end namespace clang