[Tooling] A new framework for executing clang frontend actions.

[ioeric]

Summary

This defines a clang::tooling::ToolExecutor interface that can be extended to support different execution plans including standalone execution on a given set of TUs or parallel execution on all TUs in a codebase.

In order to enable multiprocessing execution, tool actions are expected to output result into a ToolResults interface provided by executors. The ToolResults interface abstracts how results are stored e.g. in-memory for standalone executions or on-disk for large-scale execution.

New executors can be registered as ToolExecutorPlugins via the ToolExecutorPluginRegistry. CLI tools can use createExecutorFromCommandLineArgs to create a specific registered executor according to the command-line arguments.

This patch also implements StandaloneToolExecutor which has the same behavior as the current ClangTool interface, i.e. execute frontend actions on a given set of TUs. At this point, it's simply a wrapper around ClangTool at this point.

This is still experimental but expected to replace the existing ClangTool interface so that specific tools would not need to worry about execution.

[ioeric]

• Merged with origin/master

[ioeric]

• Added ToolResults interface and moved classes to right files.

[ioeric]

• Minor cleanups.

[hokein]

This is a good start, a few comments. Also I'd suggest running clang-format on this patch -- I saw a few places violate the code style.

include/clang/Tooling/CommonOptionsParser.h

89–90

... is set to true for clarify.

114

return llvm::StringRef?

include/clang/Tooling/Execution.h

22

Consider moving this comment to ToolExecutor class to make it more discoverable?

48

How about using std::pair<std::string, std::string> type here? It would make it consistent with the allKVResults below.

Also I think we can define an alias KVResult for this type.

49

nit: all

82

nit: execute

lib/Tooling/Execution.cpp

62

Consider we have two plugins, the first iteration fails, and the second one succeeds, the code looks like still treating it as an error here? And if the case where there are more than one executor being created successfully, we just return the first one.

My understanding of the createExecutorFromCommandLineArgs's behavior is to find a proper ToolExecutorPlugin and create a ToolExecutor instance.

Maybe add a command-line flag like --executor=<my_executor_name> to make it straightforward to choose which registered executor is used, and make StandaloneToolExecutor as default?

74

nit: Executor

unittests/Tooling/ToolingTest.cpp

32 ↗

(On Diff #118898)

Seems that it is unused.

638 ↗

(On Diff #118898)

any reason to make it static?

667 ↗

(On Diff #118898)

nit: size_t, the same below.

[ioeric]

• clang-format code.

[ioeric]

• Address review comments.

[ioeric]

Thanks for the review!

include/clang/Tooling/CommonOptionsParser.h

114

I'm not a big fan of StringRef as a return value. Callers can simply use StringRef s = parser.getErrorMessage(), if they want to; otherwise, they would need to worry about the live time of s in auto s = parser.getErrorMessage(), if the interface returns a StringRef.

include/clang/Tooling/Execution.h

48

It would make the the interface a bit cumbersome to use - callers would need to make pairs all the time. And converting to pairs is an unnecessary performance overhead IMO. The consistency with AllKVResults is not that important after all.

We don't have a lot usages of std::pair<std::string, std::string> yet. I'll add an alias when needed. Currently, I think it makes sense to keep the result type explicit.

49

I think for-each is a more canonical name? E.g. we have for_each loop in C++.

lib/Tooling/Execution.cpp

62

Yes, we simply return the first plugin that matches.

It's hard to ensure that all executors have mutually exclusive arguments when more executors are added; it should be up to the executors to define unique (or as-unique-as-possible) arguments. For example, a map-reduce-based executor could require a "--mr" option to be set. The factory should simply return the first one that matches.

The point of this helper function is to hide the dispatching of different executors, so I don't think requiring users to specify the executor name is sensible. For example, users should only care about whether to set --mr instead of having to know the existence of different executors.

unittests/Tooling/ToolingTest.cpp

638 ↗

(On Diff #118898)

Added a comment explaining this.

667 ↗

(On Diff #118898)

createExecutorFromCommandLineArgs and, more importantly, the underlying library functions take int, so... :(

[ioeric]

• Add a ExecutionContext class.

[arphaman]

include/clang/Tooling/CommonOptionsParser.h

120

Would it be better to have an `llvm::Error' instead of the two fields?

include/clang/Tooling/Execution.h

48

You don't need to use the llvm:: prefix for StringRef.

77

Why not unique_ptr/shared_ptr? Who owns the results?

135

Standalone is one word.

137

Maybe this class and InMemoryToolResults should be in a separate header?

[ioeric]

• Fix broken unit tests when they are run in threads in the same process.

[ioeric]

• Move StandaloneToolExecutor into a separate header; added ExecutionTest.cpp.

[ioeric]

• Addressed review comments.

[ioeric]

Thanks for the review!

include/clang/Tooling/CommonOptionsParser.h

120

It would make the code a bit more complicated... we would need to convert between Error and strings a few times here and in the user code.

include/clang/Tooling/Execution.h

77

The context creator owns the results. Here we only store a reference.

137

Moved StandaloneToolExecutor into a separate header. I am keeping InMemoryToolResults here since it might be used by other executors.

[ioeric]

• Minor cleanup in tests.

[klimek]

include/clang/Tooling/CommonOptionsParser.h

89–90

If we want error handling, why not make it a static factory that returns an ErrorOr instead?

include/clang/Tooling/Execution.h

27–28

I'd remove this line. Those tend to only get stale without anybody changing them :)

50

I think = default is the new cool way.

52–54

Why do we need to get the results via the interface? For example, in a map/reduce style setup getting the results is infeasible.

77

I think it's weird that this is virtual, but we also have a default implementation that returns something that we require in the constructor. Either make that virtual, and don't put in a default implementation, or make it non-virtual.

91

I don't think we need to annotate every unowned pointer. Pointers are unowned by default, otherwise we'd use unique_ptr.

165–167

I think for clang-refactor we'll need one level of abstraction in the middle:
We'll need to be able to say "run on all translation units referencing symbol X" - how will this fit into the design?
Also, how will code changes generally fit with this design?

[ioeric]

• Add a factory method for creating CommonOptionsParser.
• Minor cleanup in CommonOptionsParser.
• Revert CommonOptionsParser changes.
• Merge branch 'option' into arcpatch-D34272

[ioeric]

• Address review comments.

include/clang/Tooling/CommonOptionsParser.h

89–90

Done. I pulled the changes for CommonOptionsParser into a separate patch (D39042 ). PTAL

include/clang/Tooling/Execution.h

27–28

Right :)

52–54

This would enable tools to access results regardless of the underlying representation of the results.

In a map/reduce style execution, we could have an implementation that deals with fetching and reading remote files; information about remote files can be fed into the implementation from the executor that performs such execution. It might make sense to add an interface that returns the metadata about the underlying data though (e.g. URI for remote files). WDYT?

165–167

I think for clang-refactor we'll need one level of abstraction in the middle:
We'll need to be able to say "run on all translation units referencing symbol X" - how will this fit into the design?

Yes. Firstly, I think clang-refactor is responsible for getting all TUs that reference a symbols and feeding the set of TUs into the tool interface. And then we could implements a "smart" executor that picks an appropriate execution (e.g. delegate to standalone executor or map/reduce style execution) according to the number of TUs.

Also, how will code changes generally fit with this design?

Code changes will be reported by tools/actions via the ToolResults interface as key-value pairs (e.g. source location + serialized AtomicChange). After the execution, callers would have access to the results in ToolResults, or we could also provide an interface that returns the information about the results so that they can be accessed later.

[arphaman]

include/clang/Tooling/StandaloneExecution.h

2

StandaloneExecution.h?

[ioeric]

• Remove unused variable.
• Narrow supported actions to FrontendActionFactory from ToolAction.

[ioeric]

• Experimented and refined the interfaces. o Get rid of ExecutionConfig class; pass actions into executors directly. o Add ArgumentAdjuster to ExecutionContext. o Make command-line based executor selection more explicit by requiring a "--executor" option.

[ioeric]

PTAL

I experimented with our internal tools (incl. mapreduce-based execution) and modified the interfaces a bit:
o Get rid of ExecutionConfig class; pass actions into executors directly.
o Moved ArgumentAdjusters that are common to all actions into ExecutionContext.
o Make command-line based executor selection more explicit by requiring an --executor option.

[ioeric]

• Get rid of the unused 'createExecutorByName' interface.

[ioeric]

• Only expose result reporting interface from ExecutionContext so that callbacks don't have access to results.

[ioeric]

• Minor comment update.

[ioeric]

include/clang/Tooling/Execution.h

52–54

After offline discussion, I now understand the concern about having tool callbacks be able to access results.

I've changed the ExecutionContext to only expose a result reporting interface instead of returning a reference of ToolResults - callbacks are expected to only have access to ExecutionContext. The ToolResults would only be available from the executor itself.

[klimek]

include/clang/Tooling/Execution.h

76

I think the argument adjuster adjustment shouldn't be part of this interface, as the argument adjusters cannot be changed in the phase in which we want the ExecutionContext to be used.

I'd just make the argument adjusters a parameter on the constructor here (or alternatively, do not couple them in here, and just hand them around as a separate entity).

130–131

I'd put the ArgumentAdjust second (as the action is the primary thing being acted on).

[ioeric]

• Addressed review comments o Removed argument adjusting interfaces from ExecutionContext o Switched positions of Action and Adjuster parameters in the execute interfaces

[ioeric]

include/clang/Tooling/Execution.h

76

You are right. After a second thought, the adjuster here is actually only used by the executor itself (e.g. default args), and executors can maintain their own adjusters internally, so it doesn't need to be in the interface.

I've removed the adjuster interfaces from ExecutionContext. Action-specific adjusters can still be passed in through the ToolExecutor::execute(...) interface.

[klimek]

lg

[FlameTop]

Hello, I'm afraid this commit is failing on Windows.

http://lab.llvm.org:8011/console?builder=llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast&builder=llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast

FAIL: Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleActionWithResult (14523 of 35936)

• TEST 'Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleActionWithResult' FAILED ****

Note: Google Test filter = StandaloneToolTest.SimpleActionWithResult

[==========] Running 1 test from 1 test case.

[----------] Global test environment set-up.

[----------] 1 test from StandaloneToolTest

[ RUN ] StandaloneToolTest.SimpleActionWithResult

C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.src\tools\clang\unittests\Tooling\ExecutionTest.cpp(217): error: Value of: !Err

Actual: false

Expected: true

error: error reading 'C:\a.cc'

1 error generated.

Error while processing C:\a.cc.

Program aborted due to an unhandled Error:

Failed to run action.LLVMSymbolizer: error reading file: PDB Error: Unable to load PDB. Make sure the file exists and is readable. Calling loadDataForExe

#0 0x0114e867 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x45e867)

#1 0x76805422 (C:\WINDOWS\System32\ucrtbase.dll+0xa5422)

#2 0x00d67dec (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x77dec)

#3 0x011820a4 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x4920a4)

---

PASS: Extra Tools Unit Tests :: change-namespace/./ChangeNamespaceTests.exe/ChangeNamespaceTest.SimpleMoveEnum (14524 of 35936)
PASS: Extra Tools Unit Tests :: change-namespace/./ChangeNamespaceTests.exe/ChangeNamespaceTest.DontMoveForwardDeclarationInClass (14525 of 35936)
FAIL: Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleAction (14526 of 35936)

• TEST 'Clang-Unit :: Tooling/./ToolingTests.exe/StandaloneToolTest.SimpleAction' FAILED ****

Note: Google Test filter = StandaloneToolTest.SimpleAction

[==========] Running 1 test from 1 test case.

[----------] Global test environment set-up.

[----------] 1 test from StandaloneToolTest

[ RUN ] StandaloneToolTest.SimpleAction

C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.src\tools\clang\unittests\Tooling\ExecutionTest.cpp(204): error: Value of: !Err

Actual: false

Expected: true

error: error reading 'C:\a.cc'

1 error generated.

Error while processing C:\a.cc.

Program aborted due to an unhandled Error:

Failed to run action.LLVMSymbolizer: error reading file: PDB Error: Unable to load PDB. Make sure the file exists and is readable. Calling loadDataForExe

#0 0x0114e867 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x45e867)

#1 0x76805422 (C:\WINDOWS\System32\ucrtbase.dll+0xa5422)

#2 0x00d67dec (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x77dec)

#3 0x011820a4 (C:\ps4-buildslave2\llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast\llvm.obj\tools\clang\unittests\Tooling\ToolingTests.exe+0x4920a4)

---

[bjope]

I get errors when compiling due to this commit:

FAILED: tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o 
/repo/app/clang/3.6/bin/clang++  -march=corei7  -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/clang/lib/Tooling -I../tools/clang/lib/Tooling -I../tools/clang/include -Itools/clang/include -Iinclude -I../include -I/repo/app/valgrind/3.11.0/include  -fPIC -fvisibility-inlines-hidden -Werror -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fcolor-diagnostics -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3    -UNDEBUG  -fno-exceptions -fno-rtti -MMD -MT tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o -MF tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o.d -o tools/clang/lib/Tooling/CMakeFiles/clangTooling.dir/StandaloneExecution.cpp.o -c ../tools/clang/lib/Tooling/StandaloneExecution.cpp
../tools/clang/lib/Tooling/StandaloneExecution.cpp:65:11: error: unused variable 'Ret' [-Werror,-Wunused-variable]
  if (int Ret = Tool.run(Action.first.get()))
          ^
1 error generated.

include/clang/Tooling/CommonOptionsParser.h

public:
 
   /// \brief Parses command-line, initializes a compilation database.
   ///
   /// This constructor can change argc and argv contents, e.g. consume
   /// command-line options used for creating FixedCompilationDatabase.
   ///
   /// All options not belonging to \p Category become hidden.
   ///
   /// It also allows calls to set the required number of positional parameters.
   CommonOptionsParser(int &argc, const char **argv,
                       llvm::cl::OptionCategory &Category,
                       llvm::cl::NumOccurrencesFlag OccurrencesFlag,
                       const char *Overview = nullptr);
 
   /// \brief A factory method that is similar to the above constructor, except
   /// this returns an error instead exiting the program on error.
   static llvm::Expected<CommonOptionsParser>
   create(int &argc, const char **argv, llvm::cl::OptionCategory &Category,
          llvm::cl::NumOccurrencesFlag OccurrencesFlag,
          const char *Overview = nullptr);
 
   /// Returns a reference to the loaded compilations database.
   CompilationDatabase &getCompilations() {
     return *Compilations;
   }
 
   /// Returns a list of source file paths to process.
   const std::vector<std::string> &getSourcePathList() const {
     return SourcePathList;
   }
 
+  /// Returns the argument adjuster calculated from "--extra-arg" and
+  //"--extra-arg-before" options.
+  ArgumentsAdjuster getArgumentsAdjuster() { return Adjuster; }
+
   static const char *const HelpMessage;
 
 private:
   CommonOptionsParser() = default;
 
   llvm::Error init(int &argc, const char **argv,
                    llvm::cl::OptionCategory &Category,
                    llvm::cl::NumOccurrencesFlag OccurrencesFlag,
                    const char *Overview);
 
   std::unique_ptr<CompilationDatabase> Compilations;
   std::vector<std::string> SourcePathList;
+  ArgumentsAdjuster Adjuster;
 };
 
 class ArgumentsAdjustingCompilations : public CompilationDatabase {
 public:
   ArgumentsAdjustingCompilations(
       std::unique_ptr<CompilationDatabase> Compilations)
       : Compilations(std::move(Compilations)) {}
 

include/clang/Tooling/Execution.h

+//===--- Execution.h - Executing clang frontend actions -*- C++ ---------*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file defines framework for executing clang frontend actions.
+//
+//  The framework can be extended to support different execution plans including
+//  standalone execution on the given TUs or parallel execution on all TUs in
+//  the codebase.
+//
+//  In order to enable multiprocessing execution, tool actions are expected to
+//  output result into the ToolResults provided by the executor. The
+//  `ToolResults` is an interface that abstracts how results are stored e.g.
+//  in-memory for standalone execution or on-disk for large-scale execution.
+//
+//  New executors can be registered as ToolExecutorPlugins via the
+//  `ToolExecutorPluginRegistry`. CLI tools can use
+//  `createExecutorFromCommandLineArgs` to create a specific registered executor
+//  according to the command-line arguments.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_EXECUTION_H
+#define LLVM_CLANG_TOOLING_EXECUTION_H
+
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Registry.h"
+
+namespace clang {
+namespace tooling {
+
+/// \brief An abstraction for the result of a tool execution. For example, the
+/// underlying result can be in-memory or on-disk.
+///
+/// Results should be string key-value pairs. For example, a refactoring tool
+/// can use source location as key and a replacement in YAML format as value.
+class ToolResults {
+public:
+  virtual ~ToolResults() = default;
+  virtual void addResult(StringRef Key, StringRef Value) = 0;
+  virtual std::vector<std::pair<std::string, std::string>> AllKVResults() = 0;
+  virtual void forEachResult(
+      llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) = 0;
+};
+
+class InMemoryToolResults : public ToolResults {
+public:
+  void addResult(StringRef Key, StringRef Value) override;
+  std::vector<std::pair<std::string, std::string>> AllKVResults() override;
+  void forEachResult(llvm::function_ref<void(StringRef Key, StringRef Value)>
+                         Callback) override;
+
+private:
+  std::vector<std::pair<std::string, std::string>> KVResults;
+};
+
+/// \brief The context of an execution, including the information about
+/// compilation and results.
+class ExecutionContext {
+public:
+  virtual ~ExecutionContext() {}
+
+  /// \brief Initializes a context. This does not take ownership of `Results`.
+  explicit ExecutionContext(ToolResults *Results);
+
+  /// \brief Adds a KV pair to the result container of this execution.
+  void reportResult(StringRef Key, StringRef Value);
+
+  void appendArgumentsAdjuster(ArgumentsAdjuster Adjuster);
+
+  void clearArgumentsAdjuster() { ArgsAdjuster = nullptr; }
+
+  ArgumentsAdjuster getArgumentsAdjuster() const { return ArgsAdjuster; }
+
+  // Returns the source control system's revision number if applicable.
+  // Otherwise returns an empty string.
+  virtual std::string getRevision() { return ""; }
+
+  // Returns the corpus being analyzed, e.g. "llvm" for the LLVM codebase, if
+  // applicable.
+  virtual std::string getCorpus() { return ""; }
+
+  // Returns the currently processed compilation unit if available.
+  virtual std::string getCurrentCompilationUnit() { return ""; }
+
+private:
+  ToolResults *Results;
+  // This is used to adjust the compile arguments, e.g. from the compilation
+  // database, when processing a file.
+  ArgumentsAdjuster ArgsAdjuster;
+};
+
+/// \brief Interface for executing clang frontend actions.
+///
+/// This can be extended to support running tool actions in different
+/// execution mode, e.g. on a specific set of TUs or many TUs in parallel.
+///
+///  New executors can be registered as ToolExecutorPlugins via the
+///  `ToolExecutorPluginRegistry`. CLI tools can use
+///  `createExecutorFromCommandLineArgs` to create a specific registered
+///  executor according to the command-line arguments.
+class ToolExecutor {
+public:
+  virtual ~ToolExecutor() {}
+
+  /// \brief Returns the name of a specific executor.
+  virtual StringRef getExecutorName() const = 0;
+
+  /// \brief Executes each action with a corresponding arguments adjuster. This
+  /// combines the given arguments adjusters with the adjuster in the
+  /// `ExecutionContext`.
+  virtual llvm::Error
+  execute(llvm::ArrayRef<
+          std::pair<ArgumentsAdjuster, std::unique_ptr<FrontendActionFactory>>>
+              Actions) = 0;
+
+  /// \brief Convenient functions for the above `execute`.
+  ///
+  /// Executes an action with only the argument adjuster in the context.
+  llvm::Error execute(std::unique_ptr<FrontendActionFactory> Action);
+  /// Executes an action with an argument adjuster in addition to the one in the
+  /// context.
+  llvm::Error execute(ArgumentsAdjuster Adjuster,
+                      std::unique_ptr<FrontendActionFactory> Action);
+
+  /// \brief Returns a reference to the execution context.
+  ///
+  /// This should be passed to tool callbacks, and tool callbacks should report
+  /// results via the returned context.
+  virtual ExecutionContext *getExecutionContext() = 0;
+
+  /// \brief Returns a reference to the result container.
+  ///
+  /// NOTE: This should only be used after the execution finishes. Tool
+  /// callbacks should report results via `ExecutionContext` instead.
+  virtual ToolResults *getToolResults() = 0;
+
+  /// \brief Map a virtual file to be used while running the tool.
+  ///
+  /// \param FilePath The path at which the content will be mapped.
+  /// \param Content A buffer of the file's content.
+  virtual void mapVirtualFile(StringRef FilePath, StringRef Content) = 0;
+};
+
+/// \brief Interface for factories that create specific executors. This is also
+/// used as a plugin to be registered into ToolExecutorPluginRegistry.
+class ToolExecutorPlugin {
+public:
+  virtual ~ToolExecutorPlugin() {}
+
+  /// \brief Create an `ToolExecutor`.
+  ///
+  /// `OptionsParser` can be consumed (e.g. moved) if the creation succeeds.
+  virtual llvm::Expected<std::unique_ptr<ToolExecutor>>
+  create(CommonOptionsParser &OptionsParser) = 0;
+};
+
+/// \brief This creates a ToolExecutor that is in the global registry based on
+/// commandline arguments.
+///
+/// This picks the right executor based on the `--executor` option. This parses
+/// the commandline arguments with `CommonOptionsParser`, so caller does not
+/// need to parse again.
+///
+/// By default, this creates a `StandaloneToolExecutor` ("standalone") if
+/// `--executor` is not provided.
+llvm::Expected<std::unique_ptr<ToolExecutor>>
+createExecutorFromCommandLineArgs(int &argc, const char **argv,
+                                  llvm::cl::OptionCategory &Category,
+                                  const char *Overview = nullptr);
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_EXECUTION_H

include/clang/Tooling/StandaloneExecution.h

+//===--- StandaloneExecution.h - Standalone execution. -*- C++ ----------*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file defines standalone execution of clang tools.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
+#define LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H
+
+#include "clang/Tooling/Execution.h"
+
+namespace clang {
+namespace tooling {
+
+/// \brief A standalone executor that runs FrontendActions on a given set of
+/// TUs in sequence.
+class StandaloneToolExecutor : public ToolExecutor {
+public:
+  static const char *ExecutorName;
+
+  /// \brief Init with \p CompilationDatabase and the paths of all files to be
+  /// proccessed.
+  StandaloneToolExecutor(
+      const CompilationDatabase &Compilations,
+      llvm::ArrayRef<std::string> SourcePaths,
+      std::shared_ptr<PCHContainerOperations> PCHContainerOps =
+          std::make_shared<PCHContainerOperations>());
+
+  /// \brief Init with \p CommonOptionsParser. This is expected to be used by
+  /// `createExecutorFromCommandLineArgs` based on commandline options.
+  ///
+  /// The executor takes ownership of \p Options.
+  StandaloneToolExecutor(
+      CommonOptionsParser Options,
+      std::shared_ptr<PCHContainerOperations> PCHContainerOps =
+          std::make_shared<PCHContainerOperations>());
+
+  StringRef getExecutorName() const override { return ExecutorName; }
+
+  using ToolExecutor::execute;
+
+  llvm::Error
+  execute(llvm::ArrayRef<
+          std::pair<ArgumentsAdjuster, std::unique_ptr<FrontendActionFactory>>>
+              Actions) override;
+
+  /// \brief Set a \c DiagnosticConsumer to use during parsing.
+  void setDiagnosticConsumer(DiagnosticConsumer *DiagConsumer) {
+    Tool.setDiagnosticConsumer(DiagConsumer);
+  }
+
+  ExecutionContext *getExecutionContext() override { return &Context; };
+
+  ToolResults *getToolResults() override { return &Results; }
+
+  llvm::ArrayRef<std::string> getSourcePaths() const {
+    return Tool.getSourcePaths();
+  }
+
+  void mapVirtualFile(StringRef FilePath, StringRef Content) override {
+    Tool.mapVirtualFile(FilePath, Content);
+  }
+
+  /// \brief Returns the file manager used in the tool.
+  ///
+  /// The file manager is shared between all translation units.
+  FileManager &getFiles() { return Tool.getFiles(); }
+
+private:
+  // Used to store the parser when the executor is initialized with parser.
+  llvm::Optional<CommonOptionsParser> OptionsParser;
+  // FIXME: The standalone executor is currently just a wrapper of `ClangTool`.
+  // Merge `ClangTool` implementation into the this.
+  ClangTool Tool;
+  ExecutionContext Context;
+  InMemoryToolResults Results;
+};
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_STANDALONEEXECUTION_H

include/clang/Tooling/ToolExecutorPluginRegistry.h

+//===--- ToolExecutorPluginRegistry.h - -------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
+#define LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H
+
+#include "clang/Tooling/Execution.h"
+#include "llvm/Support/Registry.h"
+
+namespace clang {
+namespace tooling {
+
+typedef llvm::Registry<ToolExecutorPlugin> ToolExecutorPluginRegistry;
+
+} // end namespace tooling
+} // end namespace clang
+
+#endif // LLVM_CLANG_TOOLING_TOOLEXECUTORPLUGINREGISTRY_H

include/clang/Tooling/Tooling.h

 //  - running match/replace tools over C++ code
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLING_TOOLING_H
 #define LLVM_CLANG_TOOLING_TOOLING_H
 
 #include "clang/AST/ASTConsumer.h"
-#include "clang/Frontend/PCHContainerOperations.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/FileManager.h"
 #include "clang/Basic/LLVM.h"
 #include "clang/Driver/Util.h"
 #include "clang/Frontend/FrontendAction.h"
+#include "clang/Frontend/PCHContainerOperations.h"
 #include "clang/Lex/ModuleLoader.h"
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Option/Option.h"
 #include <memory>
 #include <string>
public:
   /// append them to ASTs.
   int buildASTs(std::vector<std::unique_ptr<ASTUnit>> &ASTs);
 
   /// \brief Returns the file manager used in the tool.
   ///
   /// The file manager is shared between all translation units.
   FileManager &getFiles() { return *Files; }
 
- private:
+  llvm::ArrayRef<std::string> getSourcePaths() const { return SourcePaths; }
+
   const CompilationDatabase &Compilations;
   std::vector<std::string> SourcePaths;
   std::shared_ptr<PCHContainerOperations> PCHContainerOps;
 
   llvm::IntrusiveRefCntPtr<vfs::OverlayFileSystem> OverlayFileSystem;
   llvm::IntrusiveRefCntPtr<vfs::InMemoryFileSystem> InMemoryFileSystem;
   llvm::IntrusiveRefCntPtr<FileManager> Files;
   // Contains a list of pairs (<file name>, <file content>).

lib/Tooling/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Option
   Support
   )
 
 add_subdirectory(Core)
 add_subdirectory(Refactoring)
 add_subdirectory(ASTDiff)
 
 add_clang_library(clangTooling
   ArgumentsAdjusters.cpp
   CommonOptionsParser.cpp
   CompilationDatabase.cpp
+  Execution.cpp
   FileMatchTrie.cpp
   FixIt.cpp
   JSONCompilationDatabase.cpp
   Refactoring.cpp
   RefactoringCallbacks.cpp
+  StandaloneExecution.cpp
   Tooling.cpp
 
   DEPENDS
   ClangDriverOptions
 
   LINK_LIBS
   clangAST
   clangASTMatchers
   clangBasic
   clangDriver
   clangFormat
   clangFrontend
   clangLex
   clangRewrite
   clangToolingCore
   )

lib/Tooling/CommonOptionsParser.cpp

if (!Compilations) {
                    << ErrorMessage << "Running without flags.\n";
       Compilations.reset(
           new FixedCompilationDatabase(".", std::vector<std::string>()));
     }
   }
   auto AdjustingCompilations =
       llvm::make_unique<ArgumentsAdjustingCompilations>(
           std::move(Compilations));
-  AdjustingCompilations->appendArgumentsAdjuster(
-      getInsertArgumentAdjuster(ArgsBefore, ArgumentInsertPosition::BEGIN));
-  AdjustingCompilations->appendArgumentsAdjuster(
+  Adjuster =
+      getInsertArgumentAdjuster(ArgsBefore, ArgumentInsertPosition::BEGIN);
+  Adjuster = combineAdjusters(
+      std::move(Adjuster),
       getInsertArgumentAdjuster(ArgsAfter, ArgumentInsertPosition::END));
+  AdjustingCompilations->appendArgumentsAdjuster(Adjuster);
   Compilations = std::move(AdjustingCompilations);
   return llvm::Error::success();
 }
 
 llvm::Expected<CommonOptionsParser> CommonOptionsParser::create(
     int &argc, const char **argv, llvm::cl::OptionCategory &Category,
     llvm::cl::NumOccurrencesFlag OccurrencesFlag, const char *Overview) {
   CommonOptionsParser Parser;

lib/Tooling/Execution.cpp

+//===- lib/Tooling/Execution.cpp - Implements tool execution framework. ---===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/ToolExecutorPluginRegistry.h"
+#include "clang/Tooling/Tooling.h"
+
+LLVM_INSTANTIATE_REGISTRY(clang::tooling::ToolExecutorPluginRegistry)
+
+namespace clang {
+namespace tooling {
+
+static llvm::cl::opt<std::string>
+    ExecutorName("executor", llvm::cl::desc("The name of the executor to use."),
+                 llvm::cl::init("standalone"));
+
+void InMemoryToolResults::addResult(StringRef Key, StringRef Value) {
+  KVResults.push_back({Key.str(), Value.str()});
+}
+
+std::vector<std::pair<std::string, std::string>>
+InMemoryToolResults::AllKVResults() {
+  return KVResults;
+}
+
+void InMemoryToolResults::forEachResult(
+    llvm::function_ref<void(StringRef Key, StringRef Value)> Callback) {
+  for (const auto &KV : KVResults) {
+    Callback(KV.first, KV.second);
+  }
+}
+
+ExecutionContext::ExecutionContext(ToolResults *Results) : Results(Results) {
+  appendArgumentsAdjuster(getClangStripOutputAdjuster());
+  appendArgumentsAdjuster(getClangSyntaxOnlyAdjuster());
+  appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
+}
+
+void ExecutionContext::reportResult(StringRef Key, StringRef Value) {
+  Results->addResult(Key, Value);
+}
+
+void ExecutionContext::appendArgumentsAdjuster(ArgumentsAdjuster Adjuster) {
+  if (!Adjuster)
+    return;
+  if (ArgsAdjuster)
+    ArgsAdjuster =
+        combineAdjusters(std::move(ArgsAdjuster), std::move(Adjuster));
+  else
+    ArgsAdjuster = std::move(Adjuster);
+}
+
+llvm::Error
+ToolExecutor::execute(std::unique_ptr<FrontendActionFactory> Action) {
+  return execute(ArgumentsAdjuster(), std::move(Action));
+}
+
+llvm::Error
+ToolExecutor::execute(ArgumentsAdjuster Adjuster,
+                      std::unique_ptr<FrontendActionFactory> Action) {
+  std::vector<
+      std::pair<ArgumentsAdjuster, std::unique_ptr<FrontendActionFactory>>>
+      Actions;
+  Actions.emplace_back(std::move(Adjuster), std::move(Action));
+  return execute(Actions);
+}
+
+llvm::Expected<std::unique_ptr<ToolExecutor>>
+createExecutorFromCommandLineArgs(int &argc, const char **argv,
+                                  llvm::cl::OptionCategory &Category,
+                                  const char *Overview) {
+  auto OptionsParser =
+      CommonOptionsParser::create(argc, argv, Category, llvm::cl::ZeroOrMore,
+                                  /*Overview=*/nullptr);
+  if (!OptionsParser)
+    return OptionsParser.takeError();
+  for (auto I = ToolExecutorPluginRegistry::begin(),
+            E = ToolExecutorPluginRegistry::end();
+       I != E; ++I) {
+    if (I->getName() != ExecutorName) {
+      continue;
+    }
+    std::unique_ptr<ToolExecutorPlugin> Plugin(I->instantiate());
+    llvm::Expected<std::unique_ptr<ToolExecutor>> Executor =
+        Plugin->create(*OptionsParser);
+    if (!Executor) {
+      return llvm::make_error<llvm::StringError>(
+          llvm::Twine("Failed to create '") + I->getName() +
+              "': " + llvm::toString(Executor.takeError()) + "\n",
+          llvm::inconvertibleErrorCode());
+    }
+    return std::move(*Executor);
+  }
+  return llvm::make_error<llvm::StringError>(
+      llvm::Twine("Executor \"") + ExecutorName + "\" is not registered.",
+      llvm::inconvertibleErrorCode());
+}
+
+} // end namespace tooling
+} // end namespace clang

lib/Tooling/StandaloneExecution.cpp

+//===- lib/Tooling/Execution.cpp - Standalone clang action execution. -----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/StandaloneExecution.h"
+#include "clang/Tooling/ToolExecutorPluginRegistry.h"
+
+namespace clang {
+namespace tooling {
+
+static llvm::Error make_string_error(const llvm::Twine &Message) {
+  return llvm::make_error<llvm::StringError>(Message,
+                                             llvm::inconvertibleErrorCode());
+}
+
+const char *StandaloneToolExecutor::ExecutorName = "StandaloneToolExecutor";
+
+static void addDefaultArgumentsAdjustersToContext(ExecutionContext *Context) {
+  Context->appendArgumentsAdjuster(getClangStripOutputAdjuster());
+  Context->appendArgumentsAdjuster(getClangSyntaxOnlyAdjuster());
+  Context->appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
+}
+
+StandaloneToolExecutor::StandaloneToolExecutor(
+    const CompilationDatabase &Compilations,
+    llvm::ArrayRef<std::string> SourcePaths,
+    std::shared_ptr<PCHContainerOperations> PCHContainerOps)
+    : Tool(Compilations, SourcePaths), Context(&Results) {
+  // Use the argument adjusters below instead of the default adjusters that
+  // come with the old `ClangTool`.
+  addDefaultArgumentsAdjustersToContext(&Context);
+  Tool.clearArgumentsAdjusters();
+}
+
+StandaloneToolExecutor::StandaloneToolExecutor(
+    CommonOptionsParser Options,
+    std::shared_ptr<PCHContainerOperations> PCHContainerOps)
+    : OptionsParser(std::move(Options)),
+      Tool(OptionsParser->getCompilations(), OptionsParser->getSourcePathList(),
+           PCHContainerOps),
+      Context(&Results) {
+  addDefaultArgumentsAdjustersToContext(&Context);
+  Tool.clearArgumentsAdjusters();
+}
+
+llvm::Error StandaloneToolExecutor::execute(
+    llvm::ArrayRef<
+        std::pair<ArgumentsAdjuster, std::unique_ptr<FrontendActionFactory>>>
+        Actions) {
+  if (Actions.empty())
+    return make_string_error("No action to execute.");
+
+  if (Actions.size() != 1)
+    return make_string_error(
+        "Only support executing exactly 1 action at this point.");
+
+  auto &Action = Actions.front();
+  if (Action.first) {
+    Tool.appendArgumentsAdjuster(Action.first);
+  }
+  if (auto Adjuster = getExecutionContext()->getArgumentsAdjuster()) {
+    Tool.appendArgumentsAdjuster(std::move(Adjuster));
+  }
+  if (int Ret = Tool.run(Action.second.get()))
+    return make_string_error("Failed to run action.");
+
+  return llvm::Error::success();
+}
+
+class StandaloneToolExecutorPlugin : public ToolExecutorPlugin {
+public:
+  llvm::Expected<std::unique_ptr<ToolExecutor>>
+  create(CommonOptionsParser &OptionsParser) override {
+    if (OptionsParser.getSourcePathList().empty())
+      return make_string_error(
+          "[StandaloneToolExecutorPlugin] No positional argument found.");
+    return llvm::make_unique<StandaloneToolExecutor>(std::move(OptionsParser));
+  }
+};
+
+// This anchor is used to force the linker to link in the generated object file
+// and thus register the plugin.
+volatile int ToolExecutorPluginAnchorSource = 0;
+
+static ToolExecutorPluginRegistry::Add<StandaloneToolExecutorPlugin>
+    X("standalone", "Runs FrontendActions on a set of files provided "
+                    "via positional arguments.");
+
+} // end namespace tooling
+} // end namespace clang

lib/Tooling/Tooling.cpp

 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Tooling/ArgumentsAdjusters.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Option/Option.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 #include <utility>
 
 #define DEBUG_TYPE "clang-tooling"

unittests/Tooling/CMakeLists.txt

 endif()
 
 add_clang_unittest(ToolingTests
   ASTSelectionTest.cpp
   CastExprTest.cpp
   CommentHandlerTest.cpp
   CompilationDatabaseTest.cpp
   DiagnosticsYamlTest.cpp
+  ExecutionTest.cpp
   FixItTest.cpp
   LexicallyOrderedRecursiveASTVisitorTest.cpp
   LookupTest.cpp
   QualTypeNamesTest.cpp
   RecursiveASTVisitorTest.cpp
   RecursiveASTVisitorTestCallVisitor.cpp
   RecursiveASTVisitorTestDeclVisitor.cpp
   RecursiveASTVisitorTestExprVisitor.cpp

unittests/Tooling/ExecutionTest.cpp

+//===- unittest/Tooling/ExecutionTest.cpp - Tool execution tests. --------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/AST/ASTConsumer.h"
+#include "clang/AST/DeclCXX.h"
+#include "clang/AST/RecursiveASTVisitor.h"
+#include "clang/Frontend/ASTUnit.h"
+#include "clang/Frontend/FrontendAction.h"
+#include "clang/Frontend/FrontendActions.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/StandaloneExecution.h"
+#include "clang/Tooling/ToolExecutorPluginRegistry.h"
+#include "clang/Tooling/Tooling.h"
+#include "gtest/gtest.h"
+#include <algorithm>
+#include <string>
+
+namespace clang {
+namespace tooling {
+
+namespace {
+
+// This traverses the AST and outputs function name as key and "1" as value for
+// each function declaration.
+class ASTConsumerWithResult
+    : public ASTConsumer,
+      public RecursiveASTVisitor<ASTConsumerWithResult> {
+public:
+  using ASTVisitor = RecursiveASTVisitor<ASTConsumerWithResult>;
+
+  explicit ASTConsumerWithResult(ExecutionContext *Context) : Context(Context) {
+    assert(Context != nullptr);
+  }
+
+  void HandleTranslationUnit(clang::ASTContext &Context) override {
+    TraverseDecl(Context.getTranslationUnitDecl());
+  }
+
+  bool TraverseFunctionDecl(clang::FunctionDecl *Decl) {
+    Context->reportResult(Decl->getNameAsString(), "1");
+    return ASTVisitor::TraverseFunctionDecl(Decl);
+  }
+
+private:
+  ExecutionContext *const Context;
+};
+
+class ReportResultAction : public ASTFrontendAction {
+public:
+  explicit ReportResultAction(ExecutionContext *Context) : Context(Context) {
+    assert(Context != nullptr);
+  }
+
+protected:
+  std::unique_ptr<clang::ASTConsumer>
+  CreateASTConsumer(clang::CompilerInstance &compiler,
+                    StringRef /* dummy */) override {
+    std::unique_ptr<clang::ASTConsumer> ast_consumer{
+        new ASTConsumerWithResult(Context)};
+    return ast_consumer;
+  }
+
+private:
+  ExecutionContext *const Context;
+};
+
+class ReportResultActionFactory : public FrontendActionFactory {
+public:
+  ReportResultActionFactory(ExecutionContext *Context) : Context(Context) {}
+  FrontendAction *create() override { return new ReportResultAction(Context); }
+
+private:
+  ExecutionContext *const Context;
+};
+
+} // namespace
+
+class TestToolExecutor : public ToolExecutor {
+public:
+  static const char *ExecutorName;
+
+  TestToolExecutor(CommonOptionsParser Options)
+      : OptionsParser(std::move(Options)) {}
+
+  StringRef getExecutorName() const override { return ExecutorName; }
+
+  llvm::Error
+  execute(llvm::ArrayRef<
+          std::pair<ArgumentsAdjuster, std::unique_ptr<FrontendActionFactory>>>)
+      override {
+    return llvm::Error::success();
+  }
+
+  ExecutionContext *getExecutionContext() override { return nullptr; };
+
+  ToolResults *getToolResults() override { return nullptr; }
+
+  llvm::ArrayRef<std::string> getSourcePaths() const {
+    return OptionsParser.getSourcePathList();
+  }
+
+  void mapVirtualFile(StringRef FilePath, StringRef Content) override {
+    VFS[FilePath] = Content;
+  }
+
+private:
+  CommonOptionsParser OptionsParser;
+  std::string SourcePaths;
+  std::map<std::string, std::string> VFS;
+};
+
+const char *TestToolExecutor::ExecutorName = "test-executor";
+
+class TestToolExecutorPlugin : public ToolExecutorPlugin {
+public:
+  llvm::Expected<std::unique_ptr<ToolExecutor>>
+  create(CommonOptionsParser &OptionsParser) override {
+    return llvm::make_unique<TestToolExecutor>(std::move(OptionsParser));
+  }
+};
+
+// This anchor is used to force the linker to link in the generated object file
+// and thus register the plugin.
+extern volatile int ToolExecutorPluginAnchorSource;
+
+static int LLVM_ATTRIBUTE_UNUSED TestToolExecutorPluginAnchorDest =
+    ToolExecutorPluginAnchorSource;
+
+static ToolExecutorPluginRegistry::Add<TestToolExecutorPlugin>
+    X("test-executor", "Plugin for TestToolExecutor.");
+
+llvm::cl::OptionCategory TestCategory("execution-test options");
+
+TEST(CreateToolExecutorTest, FailedCreateExecutorUndefinedFlag) {
+  std::vector<const char *> argv = {"prog", "--fake_flag_no_no_no", "f"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_FALSE((bool)Executor);
+  llvm::consumeError(Executor.takeError());
+}
+
+TEST(CreateToolExecutorTest, RegisterFlagsBeforeReset) {
+  llvm::cl::opt<std::string> BeforeReset(
+      "before_reset", llvm::cl::desc("Defined before reset."),
+      llvm::cl::init(""));
+
+  llvm::cl::ResetAllOptionOccurrences();
+
+  std::vector<const char *> argv = {"prog", "--before_reset=set", "f"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_TRUE((bool)Executor);
+  EXPECT_EQ(BeforeReset, "set");
+  BeforeReset.removeArgument();
+}
+
+TEST(CreateToolExecutorTest, CreateStandaloneToolExecutor) {
+  std::vector<const char *> argv = {"prog", "standalone.cpp"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_TRUE((bool)Executor);
+  EXPECT_EQ(Executor->get()->getExecutorName(),
+            StandaloneToolExecutor::ExecutorName);
+}
+
+TEST(CreateToolExecutorTest, CreateTestToolExecutor) {
+  std::vector<const char *> argv = {"prog", "test.cpp",
+                                    "--executor=test-executor"};
+  int argc = argv.size();
+  auto Executor =
+      createExecutorFromCommandLineArgs(argc, &argv[0], TestCategory);
+  ASSERT_TRUE((bool)Executor);
+  EXPECT_EQ(Executor->get()->getExecutorName(), TestToolExecutor::ExecutorName);
+}
+
+TEST(StandaloneToolTest, SynctaxOnlyActionOnSimpleCode) {
+  FixedCompilationDatabase Compilations("/", std::vector<std::string>());
+  StandaloneToolExecutor Executor(Compilations,
+                                  std::vector<std::string>(1, "/a.cc"));
+  Executor.mapVirtualFile("/a.cc", "int x = 0;");
+
+  auto Err = Executor.execute(newFrontendActionFactory<SyntaxOnlyAction>());
+  ASSERT_TRUE(!Err);
+}
+
+TEST(StandaloneToolTest, SimpleAction) {
+  FixedCompilationDatabase Compilations("/", std::vector<std::string>());
+  StandaloneToolExecutor Executor(Compilations,
+                                  std::vector<std::string>(1, "/a.cc"));
+  Executor.mapVirtualFile("/a.cc", "int x = 0;");
+
+  auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
+      new ReportResultActionFactory(Executor.getExecutionContext())));
+  ASSERT_TRUE(!Err);
+  auto KVs = Executor.getToolResults()->AllKVResults();
+  ASSERT_EQ(KVs.size(), 0u);
+}
+
+TEST(StandaloneToolTest, SimpleActionWithResult) {
+  FixedCompilationDatabase Compilations("/", std::vector<std::string>());
+  StandaloneToolExecutor Executor(Compilations,
+                                  std::vector<std::string>(1, "/a.cc"));
+  Executor.mapVirtualFile("/a.cc", "int x = 0; void f() {}");
+
+  auto Err = Executor.execute(std::unique_ptr<FrontendActionFactory>(
+      new ReportResultActionFactory(Executor.getExecutionContext())));
+  ASSERT_TRUE(!Err);
+  auto KVs = Executor.getToolResults()->AllKVResults();
+  ASSERT_EQ(KVs.size(), 1u);
+  EXPECT_EQ("f", KVs[0].first);
+  EXPECT_EQ("1", KVs[0].second);
+
+  Executor.getToolResults()->forEachResult(
+      [](StringRef, StringRef Value) { EXPECT_EQ("1", Value); });
+}
+
+} // end namespace tooling
+} // end namespace clang