Add support for importing and exporting Registry objects on Windows

[• ehsan]

Summary

On Windows, the DLL containing the registry will get its own global head
and tail variables, so the entries registered in the DLL will be
invisible to the consumer.

In order to solve this, we need to export a getter function from the
plugin DLL per registry and copy over the data inside it. This patch
adds support for this. This will be used to support clang plugins on
Windows.

Turn on LLVM_ENABLE_PLUGINS by default on Windows

[rnk]

include/llvm/Support/Registry.h

133

This should have a comment. Basically, it's copying the entire registry linked list from another DLL into the current one.

166

I think this should be LLVM_ON_WIN32. That's actually false for Cygwin and true for mingw. I'm not sure if these dllexport annotations will do the right thing with mingw, but let's find out.

175

Do you think we should have an #else block here that defines these macros to nothing on Unix, so that clients don't have to do the platform ifdefs?

[rnk]

How do you deal with importing symbols from the clang exectuable? For Chromium, we statically link our plugins into the clang binary and let the registry do its thing that way.

[• ehsan]

In D16760#342008, @rnk wrote:

How do you deal with importing symbols from the clang exectuable? For Chromium, we statically link our plugins into the clang binary and let the registry do its thing that way.

Importing symbols on Windows is a hopeless effort. We'd either need to spray everything in clang with a CLANG_API macro that resolves to dllimport/dllexport depending on what you're building, or maintain a .def file with mangled names of everything we export, both of which are terrible solutions IMO. This setup is intended to be used with static linking:

• You'll get static .lib libraries from LLVM and clang.
• To build your plugin, you pass llvm-config --cxxflags to the compiler (and similarly llvm-config --ldflags --system-libs --libs whateveryouuse to the linker) which will cause you to link against clang statically.
• The plugin DLL will create its own registry which gets imported into the main clang registry per this patch.
• clang calls into the function from the DLL (discovered through the registry) so the plugin would be able to look at clang's data structures without the need to import symbols.

This should work for code symbols, but probably not for data symbols. The Head and Tail variables here are an obvious example. As we run into such issues, we'd need to add support for them on a case by case basis.

That being said, it looks like this setup is enough for using an existing plugin which examines the AST. I haven't tried it on anything more sophisticated yet...

include/llvm/Support/Registry.h

133

Yep, will do.

166

mingw supports __declspec(dllexport) IIRC. Whether or not this specific code will work for it is of course something we'll find out. I'll fix things if it breaks mingw...

175

That sounds like a good idea. (FWIW I'm really unhappy that we needed to change the client side API for this, but sadly there is no way to avoid that.)

[rnk]

In D16760#342060, @ehsan wrote:

Importing symbols on Windows is a hopeless effort. We'd either need to spray everything in clang with a CLANG_API macro that resolves to dllimport/dllexport depending on what you're building, or maintain a .def file with mangled names of everything we export, both of which are terrible solutions IMO. This setup is intended to be used with static linking:

CMake has some new auto-export functionality that I want to look into at some point. It didn't work immediately out of the box, so I had to put it aside for now, but eventually, I would like to stop statically linking clang.exe.

include/llvm/Support/Registry.h

175

What if we kept the export side (the non-client side) the same and changed the Registry constructor to do dynamic lookup in all loaded DLLs for the exported symbol? Then we'd only have to change clang and opt.

[• ehsan]

In D16760#342227, @rnk wrote:

In D16760#342060, @ehsan wrote:

Importing symbols on Windows is a hopeless effort. We'd either need to spray everything in clang with a CLANG_API macro that resolves to dllimport/dllexport depending on what you're building, or maintain a .def file with mangled names of everything we export, both of which are terrible solutions IMO. This setup is intended to be used with static linking:

CMake has some new auto-export functionality that I want to look into at some point. It didn't work immediately out of the box, so I had to put it aside for now, but eventually, I would like to stop statically linking clang.exe.

Nice! I didn't know about that.

include/llvm/Support/Registry.h

175

Do you mean the Registry constructor running in the plugin to look at all loaded modules for a AddNodeToRegistry function and call that instead of add_node() in this patch to add its registrations to the Registry in clang.exe? (I'm not sure how opt enters the picture here.)

[• ehsan]

include/llvm/Support/Registry.h

175

I tried the following:

• Export a LLVMRegistery_${REGISTRY_NAME}_AddNode function from clang.exe.
• Get Registry::Add() to look up the function above from all of the loaded modules using DynamicLibrary to find the correct version of the function no matter whether the call is being made from clang.exe or the plugin DLL.

The problem is that in order to look up the correct function name, we need to have the name of the Registry as a string at runtime. This means that given a client doing something like FrontendPluginRegistry::Add<MyAction>() needs to somehow be passing "FrontendPluginRegistry" as a string but that's not possible without changing the client code. And if we're going to change the client code, I prefer to keep using the current approach as it's more similar to the normal way of doing things on Windows.

[• ehsan]

Addressed the review comments.

[rnk]

lgtm

cmake/modules/HandleLLVMOptions.cmake

 append_if(LLVM_BUILD_INSTRUMENTED "-fprofile-instr-generate"
   CMAKE_CXX_FLAGS
   CMAKE_C_FLAGS
   CMAKE_EXE_LINKER_FLAGS
   CMAKE_SHARED_LINKER_FLAGS)
 
 # Plugin support
 # FIXME: Make this configurable.
-if(WIN32 OR CYGWIN)
+if(CYGWIN)
   if(BUILD_SHARED_LIBS)
     set(LLVM_ENABLE_PLUGINS ON)
   else()
     set(LLVM_ENABLE_PLUGINS OFF)
   endif()
 else()
   set(LLVM_ENABLE_PLUGINS ON)
 endif()

include/llvm/Support/Registry.h

 //=== Registry.h - Linker-supported plugin registries -----------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // Defines a registry template for discovering pluggable modules.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_SUPPORT_REGISTRY_H
 #define LLVM_SUPPORT_REGISTRY_H
 
-#include "llvm/ADT/iterator_range.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/iterator_range.h"
 #include "llvm/Support/Compiler.h"
+#include "llvm/Support/DynamicLibrary.h"
 #include <memory>
 
 namespace llvm {
   /// A simple registry entry which provides only a name, description, and
   /// no-argument constructor.
   template <typename T>
   class SimpleRegistryEntry {
     const char *Name, *Desc;
private:
     friend class node;
     static node *Head, *Tail;
 
   public:
     /// Node in linked list of entries.
     ///
     class node {
       friend class iterator;
+      friend Registry<T>;
 
       node *Next;
       const entry& Val;
 
     public:
-      node(const entry& V) : Next(nullptr), Val(V) {
-        if (Tail)
-          Tail->Next = this;
-        else
-          Head = this;
-        Tail = this;
-      }
-    };
+      node(const entry &V) : Next(nullptr), Val(V) {}
+    };
+
+    static void add_node(node *N) {
+        Tail->Next = N;
+        Head = N;
+      Tail = N;
 
     /// Iterators for registry entries.
     ///
     class iterator {
       const node *Cur;
 
     public:
       explicit iterator(const node *N) : Cur(N) {}
public:
     class Add {
       entry Entry;
       node Node;
 
       static std::unique_ptr<T> CtorFn() { return make_unique<V>(); }
 
     public:
       Add(const char *Name, const char *Desc)
-        : Entry(Name, Desc, CtorFn), Node(Entry) {}
+          : Entry(Name, Desc, CtorFn), Node(Entry) {
+        add_node(&Node);
+      }
     };
+
+    /// A dynamic import facility.  This is used on Windows to
+    /// import the entries added in the plugin.
+    static void import(sys::DynamicLibrary &DL, const char *RegistryName) {
+      typedef void *(*GetRegistry)();
+      std::string Name("LLVMGetRegistry_");
+      Name.append(RegistryName);
+      GetRegistry Getter =
+          reinterpret_cast<GetRegistry>(DL.getAddressOfSymbol(Name.c_str()));
+      if (Getter) {
+        // Call the getter function in order to get the full copy of the
+        // registry defined in the plugin DLL, and copy them over to the
+        // current Registry.
+        typedef std::pair<const node *, const node *> Info;
+        Info *I = static_cast<Info *>(Getter());
+        iterator begin(I->first);
+        iterator end(I->second);
+        for (++end; begin != end; ++begin) {
+          // This Node object needs to remain alive for the
+          // duration of the program.
+          add_node(new node(*begin));
+        }
+      }
+    }
+
+    /// Retrieve the data to be passed across DLL boundaries when
+    /// importing registries from another DLL on Windows.
+    static void *exportRegistry() {
+      static std::pair<const node *, const node *> Info(Head, Tail);
+      return &Info;
+    }
   };
 
   
   // Since these are defined in a header file, plugins must be sure to export
   // these symbols.
   template <typename T>
   typename Registry<T>::node *Registry<T>::Head;
 
   template <typename T>
   typename Registry<T>::node *Registry<T>::Tail;
 } // end namespace llvm
 
+#ifdef LLVM_ON_WIN32
+#define LLVM_EXPORT_REGISTRY(REGISTRY_CLASS)                                   \
+  extern "C" {                                                                 \
+  __declspec(dllexport) void *__cdecl LLVMGetRegistry_##REGISTRY_CLASS() {     \
+    return REGISTRY_CLASS::exportRegistry();                                   \
+  }                                                                            \
+  }
+#define LLVM_IMPORT_REGISTRY(REGISTRY_CLASS, DL)                               \
+  REGISTRY_CLASS::import(DL, #REGISTRY_CLASS)
+#else
+#define LLVM_EXPORT_REGISTRY(REGISTRY_CLASS)
+#define LLVM_IMPORT_REGISTRY(REGISTRY_CLASS, DL)
+#endif
+
 #endif // LLVM_SUPPORT_REGISTRY_H