[Orc] Use extensible RTTI for the orc::ObjectLayer class hierarchy

[sgraenitz]

Summary

So far we had no way to distinguish between JITLink and RuntimeDyld in lli. Instead, we used implicit knowledge that RuntimeDyld would be used for linking ELF. In order to get D97337 to work with lli though, we have to move on and allow JITLink for ELF. This patch uses extensible RTTI to allow external clients to add their own layers without touching the LLVM sources.

[lhames]

We should use ExtensibleRTTI.h for this. Performance is a non-issue here, and I would like clients to be able to supply their own reference linking layers (e.g. one that just calls out to ld then dlopens).

[sgraenitz]

Thanks for having a look. Yes, extensibility is a good point. I switched it to the ExtensibleRTTI implementation and it works well. It's the first use-case in LLVM so far: https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html#rtti-for-open-class-hierarchies

I have to admit I am not very happy with the CRTP approach in there. I hacked forward a bit to look for an alternative and it is possible: https://reviews.llvm.org/differential/diff/326409/#change-EaiHppkLE6Ki, but it's not in a state where it could land any time soon and I don't have the capacity to get it ready for review now.

[sgraenitz]

Switch to extensible RTTI

[dblaikie]

In D97338#2587980, @sgraenitz wrote:

Thanks for having a look. Yes, extensibility is a good point. I switched it to the ExtensibleRTTI implementation and it works well. It's the first use-case in LLVM so far: https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html#rtti-for-open-class-hierarchies

I have to admit I am not very happy with the CRTP approach in there.

Could you describe more what you find off-putting about it?

I hacked forward a bit to look for an alternative and it is possible: https://reviews.llvm.org/differential/diff/326409/#change-EaiHppkLE6Ki, but it's not in a state where it could land any time soon and I don't have the capacity to get it ready for review now.

This alternative, at least, looks like it uses an extra non-static member variable, compared to the original use of a static member. That's a fairly significant tradeoff/cost to add in terms of extra size to every derived object.

[lhames]

In D97338#2588010, @dblaikie wrote:

This alternative, at least, looks like it uses an extra non-static member variable, compared to the original use of a static member. That's a fairly significant tradeoff/cost to add in terms of extra size to every derived object.

Yeah -- We considered designs using a non-static type member at the time, but decided against them for this reason. I haven't come up with a good solution that avoids CRTP and static members yet.

[sgraenitz]

In D97338#2588010, @dblaikie wrote:

Could you describe more what you find off-putting about it?

First of all I think the current implementation is reasonable and pragmatic. CRTP adds artificial layers to the class hierarchy, which can be a pain for the reader and the debugger. Maybe it's personal preference, years ago I worked on a project that used CRTP heavily and it was a pain (pre C++14 inheriting ctors).

This alternative, at least, looks like it uses an extra non-static member variable, compared to the original use of a static member. That's a fairly significant tradeoff/cost to add in terms of extra size to every derived object.

Well, I considered the idea promising. For each instance overhead is "length of inheritance chain" times "1 direct call for construction and 1 byte memory (plus alignment)" plus "one pointer in the root". Bad for small payloads and inner loops of course, but ORC ObjectLayers... Would be great, if it was possible with only the one extra member, but I didn't get rid of the extra using and that makes it odd. Another major concern is compile time overhead and additional archive size, as the whole template machinery gets pulled into Casting.h too. So, I guess there is not much coming out of my hack for now. Maybe on a lazy Sunday I get back to it :)

llvm/include/llvm/ExecutionEngine/Orc/Layer.h

 
 #ifndef LLVM_EXECUTIONENGINE_ORC_LAYER_H
 #define LLVM_EXECUTIONENGINE_ORC_LAYER_H
 
 #include "llvm/ExecutionEngine/Orc/Core.h"
 #include "llvm/ExecutionEngine/Orc/Mangling.h"
 #include "llvm/ExecutionEngine/Orc/ThreadSafeModule.h"
 #include "llvm/IR/Module.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/Support/MemoryBuffer.h"
 
 namespace llvm {
 namespace orc {
 
 /// IRMaterializationUnit is a convenient base class for MaterializationUnits
 /// wrapping LLVM IR. Represents materialization responsibility for all symbols
 /// in the given module. If symbols are overridden by other definitions, then
private:
   void materialize(std::unique_ptr<MaterializationResponsibility> R) override;
 
   IRLayer &L;
 };
 
 /// Interface for Layers that accept object files.
 class ObjectLayer {
 public:
-  ObjectLayer(ExecutionSession &ES);
+  enum class Kind {
+    JITLink = 1,
+    RuntimeDyld = 2,
+    Transform = 3,
+  };
+
+  Kind getKind() const { return ObjectLayerKind; }
+
+  ObjectLayer(ExecutionSession &ES, Kind K);
   virtual ~ObjectLayer();
 
   /// Returns the execution session for this layer.
   ExecutionSession &getExecutionSession() { return ES; }
 
   /// Adds a MaterializationUnit representing the given IR to the given
   /// JITDylib.
   virtual Error add(ResourceTrackerSP RT, std::unique_ptr<MemoryBuffer> O);
 
   Error add(JITDylib &JD, std::unique_ptr<MemoryBuffer> O) {
     return add(JD.getDefaultResourceTracker(), std::move(O));
   }
 
   /// Emit should materialize the given IR.
   virtual void emit(std::unique_ptr<MaterializationResponsibility> R,
                     std::unique_ptr<MemoryBuffer> O) = 0;
 
 private:
   ExecutionSession &ES;
+  const Kind ObjectLayerKind;
 };
 
 /// Materializes the given object file (represented by a MemoryBuffer
 /// instance) by calling 'emit' on the given ObjectLayer.
 class BasicObjectLayerMaterializationUnit : public MaterializationUnit {
 public:
   static Expected<std::unique_ptr<BasicObjectLayerMaterializationUnit>>
   Create(ObjectLayer &L, std::unique_ptr<MemoryBuffer> O);

llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h

public:
   /// co-existence with RTDyldObjectLinkingLayer (which also owns its
   /// allocators).
   ObjectLinkingLayer(ExecutionSession &ES,
                      std::unique_ptr<jitlink::JITLinkMemoryManager> MemMgr);
 
   /// Destruct an ObjectLinkingLayer.
   ~ObjectLinkingLayer();
 
+  static bool classof(const ObjectLayer *Layer) {
+    return Layer->getKind() == Kind::JITLink;
+  }
+
   /// Set an object buffer return function. By default object buffers are
   /// deleted once the JIT has linked them. If a return function is set then
   /// it will be called to transfer ownership of the buffer instead.
   void setReturnObjectBuffer(ReturnObjectBufferFunction ReturnObjectBuffer) {
     this->ReturnObjectBuffer = std::move(ReturnObjectBuffer);
   }
 
   /// Add a pass-config modifier.

llvm/include/llvm/ExecutionEngine/Orc/ObjectTransformLayer.h

 public:
   using TransformFunction =
       std::function<Expected<std::unique_ptr<MemoryBuffer>>(
           std::unique_ptr<MemoryBuffer>)>;
 
   ObjectTransformLayer(ExecutionSession &ES, ObjectLayer &BaseLayer,
                        TransformFunction Transform = TransformFunction());
 
+  static bool classof(const ObjectLayer *Layer) {
+    return Layer->getKind() == Kind::Transform;
+  }
+
   void emit(std::unique_ptr<MaterializationResponsibility> R,
             std::unique_ptr<MemoryBuffer> O) override;
 
   void setTransform(TransformFunction Transform) {
     this->Transform = std::move(Transform);
   }
 
 private:
   ObjectLayer &BaseLayer;
   TransformFunction Transform;
 };
 
 } // end namespace orc
 } // end namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_ORC_OBJECTTRANSFORMLAYER_H

llvm/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h

public:
 
   /// Construct an ObjectLinkingLayer with the given NotifyLoaded,
   ///        and NotifyEmitted functors.
   RTDyldObjectLinkingLayer(ExecutionSession &ES,
                            GetMemoryManagerFunction GetMemoryManager);
 
   ~RTDyldObjectLinkingLayer();
 
+  static bool classof(const ObjectLayer *Layer) {
+    return Layer->getKind() == Kind::RuntimeDyld;
+  }
+
   /// Emit the object.
   void emit(std::unique_ptr<MaterializationResponsibility> R,
             std::unique_ptr<MemoryBuffer> O) override;
 
   /// Set the NotifyLoaded callback.
   RTDyldObjectLinkingLayer &setNotifyLoaded(NotifyLoadedFunction NotifyLoaded) {
     this->NotifyLoaded = std::move(NotifyLoaded);
     return *this;

llvm/lib/ExecutionEngine/Orc/Layer.cpp

#endif // NDEBUG
   LLVM_DEBUG(ES.runSessionLocked(
       [&]() { dbgs() << "Emitting, for " << N << ", " << *this << "\n"; }););
   L.emit(std::move(R), std::move(TSM));
   LLVM_DEBUG(ES.runSessionLocked([&]() {
     dbgs() << "Finished emitting, for " << N << ", " << *this << "\n";
   }););
 }
 
-ObjectLayer::ObjectLayer(ExecutionSession &ES) : ES(ES) {}
+ObjectLayer::ObjectLayer(ExecutionSession &ES, Kind K)
+    : ES(ES), ObjectLayerKind(K) {}
 
 ObjectLayer::~ObjectLayer() {}
 
 Error ObjectLayer::add(ResourceTrackerSP RT, std::unique_ptr<MemoryBuffer> O) {
   assert(RT && "RT can not be null");
   auto ObjMU = BasicObjectLayerMaterializationUnit::Create(*this, std::move(O));
   if (!ObjMU)
     return ObjMU.takeError();

llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp

private:
   DenseMap<SymbolStringPtr, SymbolNameSet> ExternalNamedSymbolDeps;
   DenseMap<SymbolStringPtr, SymbolNameSet> InternalNamedSymbolDeps;
 };
 
 ObjectLinkingLayer::Plugin::~Plugin() {}
 
 ObjectLinkingLayer::ObjectLinkingLayer(ExecutionSession &ES,
                                        JITLinkMemoryManager &MemMgr)
-    : ObjectLayer(ES), MemMgr(MemMgr) {
+    : ObjectLayer(ES, Kind::JITLink), MemMgr(MemMgr) {
   ES.registerResourceManager(*this);
 }
 
 ObjectLinkingLayer::ObjectLinkingLayer(
     ExecutionSession &ES, std::unique_ptr<JITLinkMemoryManager> MemMgr)
-    : ObjectLayer(ES), MemMgr(*MemMgr), MemMgrOwnership(std::move(MemMgr)) {
+    : ObjectLayer(ES, Kind::JITLink), MemMgr(*MemMgr),
+      MemMgrOwnership(std::move(MemMgr)) {
   ES.registerResourceManager(*this);
 }
 
 ObjectLinkingLayer::~ObjectLinkingLayer() {
   assert(Allocs.empty() && "Layer destroyed with resources still attached");
   getExecutionSession().deregisterResourceManager(*this);
 }
 

llvm/lib/ExecutionEngine/Orc/ObjectTransformLayer.cpp

 //===---------- ObjectTransformLayer.cpp - Object Transform Layer ---------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ExecutionEngine/Orc/ObjectTransformLayer.h"
 #include "llvm/Support/MemoryBuffer.h"
 
 namespace llvm {
 namespace orc {
 
 ObjectTransformLayer::ObjectTransformLayer(ExecutionSession &ES,
-                                            ObjectLayer &BaseLayer,
-                                            TransformFunction Transform)
-    : ObjectLayer(ES), BaseLayer(BaseLayer), Transform(std::move(Transform)) {}
+    : ObjectLayer(ES, Kind::Transform), BaseLayer(BaseLayer),
+      Transform(std::move(Transform)) {}
 
 void ObjectTransformLayer::emit(
     std::unique_ptr<MaterializationResponsibility> R,
     std::unique_ptr<MemoryBuffer> O) {
   assert(O && "Module must not be null");
 
   // If there is a transform set then apply it.
   if (Transform) {

llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp

 
 } // end anonymous namespace
 
 namespace llvm {
 namespace orc {
 
 RTDyldObjectLinkingLayer::RTDyldObjectLinkingLayer(
     ExecutionSession &ES, GetMemoryManagerFunction GetMemoryManager)
-    : ObjectLayer(ES), GetMemoryManager(GetMemoryManager) {
+    : ObjectLayer(ES, Kind::RuntimeDyld), GetMemoryManager(GetMemoryManager) {
   ES.registerResourceManager(*this);
 }
 
 RTDyldObjectLinkingLayer::~RTDyldObjectLinkingLayer() {
   assert(MemMgrs.empty() && "Layer destroyed with resources still attached");
 }
 
 void RTDyldObjectLinkingLayer::emit(

llvm/tools/lli/lli.cpp

case LLJITPlatform::MachO:
       break;
     default:
       llvm_unreachable("Unrecognized platform value");
     }
   }
 
   auto J = ExitOnErr(Builder.create());
 
-  if (TT->isOSBinFormatELF())
-    static_cast<llvm::orc::RTDyldObjectLinkingLayer &>(J->getObjLinkingLayer())
-        .registerJITEventListener(
-            *JITEventListener::createGDBRegistrationListener());
+  if (TT->isOSBinFormatELF()) {
+    if (auto *ObjLayer =
+            dyn_cast<orc::RTDyldObjectLinkingLayer>(&J->getObjLinkingLayer()))
+      ObjLayer->registerJITEventListener(
+  }
 
   if (PerModuleLazy)
     J->setPartitionFunction(orc::CompileOnDemandLayer::compileWholeModule);
 
   auto Dump = createDebugDumper();
 
   J->getIRTransformLayer().setTransform(
       [&](orc::ThreadSafeModule TSM,