Make it possible for AST plugins to enable themselves by default

[john.brawn]

Summary

Currently when an AST plugin is loaded it must then be enabled by passing -plugin pluginname or -add-plugin pluginname to the -cc1 command line. This patch adds a method to PluginASTAction which allows it to declare that the action happens before, instead of, or after the main AST action, plus the relevant changes to make the plugin action happen at that time automatically.

Note: patch was written before I noticed D15729, and probably needs adjusting to work with that.

[klimek]

+benjamin, who has done work in the area

[john.brawn]

Looking at D15729 I think there's no conflict between what this patch is doing and what that patch is doing. If a plugin returns AddBeforeMainAction or AddAfterMainAction then when ASTUnit in libclang loads that plugin it should get automatically added in FrontendAction::CreateWrappedASTConsumer, which is what I would expect should happen. There's a question of whether lines 199-210 of ExecuteCompilerInvocation.cpp should get moved into the CompilerInstance::loadPlugins function that D15729 adds and I think the answer there is no - looking at ASTUnit it doesn't seem to use getFrontendOpts().ProgramAction at all, so there's no point setting it based on a plugin.

[bkramer]

This looks fine to me.

cfe/trunk/docs/ClangPlugins.rst

 Let's look at an example plugin that prints top-level function names.  This
 example is checked into the clang repository; please take a look at
 the `latest version of PrintFunctionNames.cpp
 <http://llvm.org/viewvc/llvm-project/cfe/trunk/examples/PrintFunctionNames/PrintFunctionNames.cpp?view=markup>`_.
 
 Running the plugin
 ==================
 
+
+Using the cc1 command line
+--------------------------
+
 To run a plugin, the dynamic library containing the plugin registry must be
 loaded via the :option:`-load` command line option. This will load all plugins
 that are registered, and you can select the plugins to run by specifying the
 :option:`-plugin` option. Additional parameters for the plugins can be passed with
 :option:`-plugin-arg-<plugin-name>`.
 
 Note that those options must reach clang's cc1 process. There are two
 ways to do so:
$ clang++ -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS \
             -I$BD/tools/clang/include -Itools/clang/include -I$BD/include -Iinclude \
             tools/clang/tools/clang-check/ClangCheck.cpp -fsyntax-only \
             -Xclang -load -Xclang $BD/lib/PrintFunctionNames.so -Xclang \
             -plugin -Xclang print-fns
 
 Also see the print-function-name plugin example's
 `README <http://llvm.org/viewvc/llvm-project/cfe/trunk/examples/PrintFunctionNames/README.txt?view=markup>`_
 
+
+Using the clang command line
+----------------------------
+
+Using :option:`-fplugin=plugin` on the clang command line passes the plugin
+through as an argument to :option:`-load` on the cc1 command line. If the plugin
+class implements the ``getActionType`` method then the plugin is run
+automatically. For example, to run the plugin automatically after the main AST
+action (i.e. the same as using :option:`-add-plugin`):
+
+.. code-block:: c++
+
+  // Automatically run the plugin after the main AST action
+  PluginASTAction::ActionType getActionType() override {
+    return AddAfterMainAction;
+  }

cfe/trunk/examples/AnnotateFunctions/AnnotateFunctions.cpp

+//===- AnnotateFunctions.cpp ----------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Example clang plugin which adds an annotation to every function.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Frontend/FrontendPluginRegistry.h"
+#include "clang/AST/AST.h"
+#include "clang/AST/ASTConsumer.h"
+using namespace clang;
+
+namespace {
+
+class AnnotateFunctionsConsumer : public ASTConsumer {
+public:
+  bool HandleTopLevelDecl(DeclGroupRef DG) override {
+    for (auto D : DG)
+      if (FunctionDecl *FD = dyn_cast<FunctionDecl>(D))
+        FD->addAttr(AnnotateAttr::CreateImplicit(FD->getASTContext(),
+                                                 "example_annotation"));
+    return true;
+  }
+};
+
+class AnnotateFunctionsAction : public PluginASTAction {
+public:
+  std::unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &CI,
+                                                 llvm::StringRef) override {
+    return llvm::make_unique<AnnotateFunctionsConsumer>();
+  }
+
+  bool ParseArgs(const CompilerInstance &CI,
+                 const std::vector<std::string> &args) override {
+    return true;
+  }
+
+  PluginASTAction::ActionType getActionType() override {
+    return AddBeforeMainAction;
+  }
+};
+
+}
+
+static FrontendPluginRegistry::Add<AnnotateFunctionsAction>
+X("annotate-fns", "annotate functions");

cfe/trunk/examples/AnnotateFunctions/CMakeLists.txt

+add_llvm_loadable_module(AnnotateFunctions AnnotateFunctions.cpp)
+
+if(LLVM_ENABLE_PLUGINS AND (WIN32 OR CYGWIN))
+  target_link_libraries(AnnotateFunctions ${cmake_2_8_12_PRIVATE}
+    clangAST
+    clangFrontend
+    LLVMSupport
+    )
+endif()

cfe/trunk/examples/CMakeLists.txt

 if(NOT CLANG_BUILD_EXAMPLES)
   set_property(DIRECTORY PROPERTY EXCLUDE_FROM_ALL ON)
   set(EXCLUDE_FROM_ALL ON)
 endif()
 
 if(CLANG_ENABLE_STATIC_ANALYZER)
 add_subdirectory(analyzer-plugin)
 endif()
 add_subdirectory(clang-interpreter)
 add_subdirectory(PrintFunctionNames)
+add_subdirectory(AnnotateFunctions)

cfe/trunk/include/clang/Frontend/FrontendAction.h

public:
   /// \brief Parse the given plugin command line arguments.
   ///
   /// \param CI - The compiler instance, for use in reporting diagnostics.
   /// \return True if the parsing succeeded; otherwise the plugin will be
   /// destroyed and no action run. The plugin is responsible for using the
   /// CompilerInstance's Diagnostic object to report errors.
   virtual bool ParseArgs(const CompilerInstance &CI,
                          const std::vector<std::string> &arg) = 0;
+
+  enum ActionType {
+    Cmdline,             //< Action is determined by the cc1 command-line
+    ReplaceAction,       //< Replace the main action
+    AddBeforeMainAction, //< Execute the action before the main action
+    AddAfterMainAction   //< Execute the action after the main action
+  };
+  /// \brief Get the action type for this plugin
+  ///
+  /// \return The action type. If the type is Cmdline then by default the
+  /// plugin does nothing and what it does is determined by the cc1
+  /// command-line.
+  virtual ActionType getActionType() { return Cmdline; }
 };
 
 /// \brief Abstract base class to use for preprocessor-based frontend actions.
 class PreprocessorFrontendAction : public FrontendAction {
 protected:
   /// \brief Provide a default implementation which returns aborts;
   /// this method should never be called by FrontendAction clients.
   std::unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &CI,

cfe/trunk/include/clang/Frontend/FrontendOptions.h

 #define LLVM_CLANG_FRONTEND_FRONTENDOPTIONS_H
 
 #include "clang/Frontend/CommandLineSourceLoc.h"
 #include "clang/Serialization/ModuleFileExtension.h"
 #include "clang/Sema/CodeCompleteOptions.h"
 #include "llvm/ADT/StringRef.h"
 #include <string>
 #include <vector>
+#include <unordered_map>
 
 namespace llvm {
 class MemoryBuffer;
 }
 
 namespace clang {
 
 namespace frontend {
public:
   ParsedSourceLocation CodeCompletionAt;
 
   /// The frontend action to perform.
   frontend::ActionKind ProgramAction;
 
   /// The name of the action to run when using a plugin action.
   std::string ActionName;
 
-  /// Args to pass to the plugin
-  std::vector<std::string> PluginArgs;
+  /// Args to pass to the plugins
+  std::unordered_map<std::string,std::vector<std::string>> PluginArgs;
 
   /// The list of plugin actions to run in addition to the normal action.
   std::vector<std::string> AddPluginActions;
 
-  /// Args to pass to the additional plugins
-  std::vector<std::vector<std::string> > AddPluginArgs;
-
   /// The list of plugins to load.
   std::vector<std::string> Plugins;
 
   /// The list of module file extensions.
   std::vector<IntrusiveRefCntPtr<ModuleFileExtension>> ModuleFileExtensions;
 
   /// \brief The list of module map files to load before processing the input.
   std::vector<std::string> ModuleMapFiles;

cfe/trunk/lib/Frontend/CompilerInvocation.cpp

case OPT_Eonly:
       Opts.ProgramAction = frontend::RunPreprocessorOnly; break;
     }
   }
 
   if (const Arg* A = Args.getLastArg(OPT_plugin)) {
     Opts.Plugins.emplace_back(A->getValue(0));
     Opts.ProgramAction = frontend::PluginAction;
     Opts.ActionName = A->getValue();
-
-    for (const Arg *AA : Args.filtered(OPT_plugin_arg))
-      if (AA->getValue(0) == Opts.ActionName)
-        Opts.PluginArgs.emplace_back(AA->getValue(1));
   }
-
   Opts.AddPluginActions = Args.getAllArgValues(OPT_add_plugin);
-  Opts.AddPluginArgs.resize(Opts.AddPluginActions.size());
-  for (int i = 0, e = Opts.AddPluginActions.size(); i != e; ++i)
-    for (const Arg *A : Args.filtered(OPT_plugin_arg))
-      if (A->getValue(0) == Opts.AddPluginActions[i])
-        Opts.AddPluginArgs[i].emplace_back(A->getValue(1));
+  for (const Arg *AA : Args.filtered(OPT_plugin_arg))
+    Opts.PluginArgs[AA->getValue(0)].emplace_back(AA->getValue(1));
 
   for (const std::string &Arg :
          Args.getAllArgValues(OPT_ftest_module_file_extension_EQ)) {
     std::string BlockName;
     unsigned MajorVersion;
     unsigned MinorVersion;
     bool Hashed;
     std::string UserInfo;

cfe/trunk/lib/Frontend/FrontendAction.cpp

 
 std::unique_ptr<ASTConsumer>
 FrontendAction::CreateWrappedASTConsumer(CompilerInstance &CI,
                                          StringRef InFile) {
   std::unique_ptr<ASTConsumer> Consumer = CreateASTConsumer(CI, InFile);
   if (!Consumer)
     return nullptr;
 
-  if (CI.getFrontendOpts().AddPluginActions.size() == 0)
+  // If there are no registered plugins we don't need to wrap the consumer
+  if (FrontendPluginRegistry::begin() == FrontendPluginRegistry::end())
     return Consumer;
 
-  // Make sure the non-plugin consumer is first, so that plugins can't
-  // modifiy the AST.
+  // Collect the list of plugins that go before the main action (in Consumers)
+  // or after it (in AfterConsumers)
   std::vector<std::unique_ptr<ASTConsumer>> Consumers;
-  Consumers.push_back(std::move(Consumer));
-
-  for (size_t i = 0, e = CI.getFrontendOpts().AddPluginActions.size();
-       i != e; ++i) { 
-    // This is O(|plugins| * |add_plugins|), but since both numbers are
-    // way below 50 in practice, that's ok.
-    for (FrontendPluginRegistry::iterator
-        it = FrontendPluginRegistry::begin(),
+  std::vector<std::unique_ptr<ASTConsumer>> AfterConsumers;
+  for (FrontendPluginRegistry::iterator it = FrontendPluginRegistry::begin(),
                                         ie = FrontendPluginRegistry::end();
-        it != ie; ++it) {
-      if (it->getName() != CI.getFrontendOpts().AddPluginActions[i])
-        continue;
-      std::unique_ptr<PluginASTAction> P = it->instantiate();
-      if (P->ParseArgs(CI, CI.getFrontendOpts().AddPluginArgs[i]))
-        Consumers.push_back(P->CreateASTConsumer(CI, InFile));
+    PluginASTAction::ActionType ActionType = P->getActionType();
+    if (ActionType == PluginASTAction::Cmdline) {
+      // This is O(|plugins| * |add_plugins|), but since both numbers are
+      // way below 50 in practice, that's ok.
+      for (size_t i = 0, e = CI.getFrontendOpts().AddPluginActions.size();
+           i != e; ++i) {
+        if (it->getName() == CI.getFrontendOpts().AddPluginActions[i]) {
+          ActionType = PluginASTAction::AddAfterMainAction;
+          break;
+        }
+      }
+    }
+    if ((ActionType == PluginASTAction::AddBeforeMainAction ||
+         ActionType == PluginASTAction::AddAfterMainAction) &&
+        P->ParseArgs(CI, CI.getFrontendOpts().PluginArgs[it->getName()])) {
+      std::unique_ptr<ASTConsumer> PluginConsumer = P->CreateASTConsumer(CI, InFile);
+      if (ActionType == PluginASTAction::AddBeforeMainAction) {
+        Consumers.push_back(std::move(PluginConsumer));
+      } else {
+        AfterConsumers.push_back(std::move(PluginConsumer));
       }
     }
+  }
+
+  // Add to Consumers the main consumer, then all the plugins that go after it
+  Consumers.push_back(std::move(Consumer));
+  for (auto &C : AfterConsumers) {
+    Consumers.push_back(std::move(C));
+  }
 
   return llvm::make_unique<MultiplexConsumer>(std::move(Consumers));
 }
 
 bool FrontendAction::BeginSourceFile(CompilerInstance &CI,
                                      const FrontendInputFile &Input) {
   assert(!Instance && "Already processing a source file!");
   assert(!Input.isEmpty() && "Unexpected empty filename!");

cfe/trunk/lib/FrontendTool/ExecuteCompilerInvocation.cpp

CreateFrontendBaseAction(CompilerInstance &CI) {
   case VerifyPCH:              return llvm::make_unique<VerifyPCHAction>();
 
   case PluginAction: {
     for (FrontendPluginRegistry::iterator it =
            FrontendPluginRegistry::begin(), ie = FrontendPluginRegistry::end();
          it != ie; ++it) {
       if (it->getName() == CI.getFrontendOpts().ActionName) {
         std::unique_ptr<PluginASTAction> P(it->instantiate());
-        if (!P->ParseArgs(CI, CI.getFrontendOpts().PluginArgs))
+        if ((P->getActionType() != PluginASTAction::ReplaceAction &&
+             P->getActionType() != PluginASTAction::Cmdline) ||
+            !P->ParseArgs(CI, CI.getFrontendOpts().PluginArgs[it->getName()]))
           return nullptr;
         return std::move(P);
       }
     }
 
     CI.getDiagnostics().Report(diag::err_fe_invalid_plugin_name)
       << CI.getFrontendOpts().ActionName;
     return nullptr;
for (unsigned i = 0,
          e = Clang->getFrontendOpts().Plugins.size(); i != e; ++i) {
     const std::string &Path = Clang->getFrontendOpts().Plugins[i];
     std::string Error;
     if (llvm::sys::DynamicLibrary::LoadLibraryPermanently(Path.c_str(), &Error))
       Clang->getDiagnostics().Report(diag::err_fe_unable_to_load_plugin)
         << Path << Error;
   }
 
+  // Check if any of the loaded plugins replaces the main AST action
+  for (FrontendPluginRegistry::iterator it = FrontendPluginRegistry::begin(),
+                                        ie = FrontendPluginRegistry::end();
+       it != ie; ++it) {
+    std::unique_ptr<PluginASTAction> P(it->instantiate());
+    if (P->getActionType() == PluginASTAction::ReplaceAction) {
+      Clang->getFrontendOpts().ProgramAction = clang::frontend::PluginAction;
+      Clang->getFrontendOpts().ActionName = it->getName();
+      break;
+    }
+  }
+
   // Honor -mllvm.
   //
   // FIXME: Remove this, one day.
   // This should happen AFTER plugins have been loaded!
   if (!Clang->getFrontendOpts().LLVMArgs.empty()) {
     unsigned NumArgs = Clang->getFrontendOpts().LLVMArgs.size();
     auto Args = llvm::make_unique<const char*[]>(NumArgs + 2);
     Args[0] = "clang (LLVM option parsing)";

cfe/trunk/test/Frontend/plugin-annotate-functions.c

+// RUN: %clang -fplugin=%llvmshlibdir/AnnotateFunctions%pluginext -emit-llvm -S %s -o - | FileCheck %s
+// REQUIRES: plugins, examples
+
+// CHECK: [[STR_VAR:@.+]] = private unnamed_addr constant [19 x i8] c"example_annotation\00"
+// CHECK: @llvm.global.annotations = {{.*}}@fn1{{.*}}[[STR_VAR]]{{.*}}@fn2{{.*}}[[STR_VAR]]
+void fn1() { }
+void fn2() { }