[ORC] Fix weak hidden symbols failure on PPC with runtimedyld

[sunho]

Summary

Fix "JIT session error: Symbols not found: [ DW.ref.__gxx_personality_v0 ] error" which happens when trying to use exceptions on ppc linux. To do this, it expands AutoClaimSymbols option in RTDyldObjectLinkingLayer to also claim weak symbols before they are tried to be resovled. In ppc linux, DW.ref symbols is emitted as weak hidden symbols in the later stage of MC pipeline. This means when using IRLayer (i.e. LLJIT), IRLayer will not claim responsibility for such symbols and RuntimeDyld will skip defining this symbol even though it couldn't resolve corresponding external symbol.

[sunho]

Expand existing option instead.

[v.g.vassilev]

This LGTM but I'd wait for @lhames.

[sgraenitz]

Thanks for the patch! I think this is the right way forward.

llvm/lib/ExecutionEngine/Orc/LLJIT.cpp

752

What about further conditions like isOSLinux() or isOSBinFormatELF()? The issue seems quite specific and AutoClaimResponsibility should be used with care. Intuitively, it seems we should tightening it to the exact platform as much as we can. It would also reduce the risk of breaking legacy use-cases downstream (which is the majority of RuntimeDyld users I think).

Are there good reasons why we wouldn't?

llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp

154

SymbolStringPool::intern() involves a mutex lock and a string-map traversal. Might be worth doing it only once where ever we can.

[sunho]

Address comments.

[sunho]

llvm/lib/ExecutionEngine/Orc/LLJIT.cpp

752

thanks for review! Yes, there was no paticular reason for not restricting the target.

[sgraenitz]

Sorry for the delay, LGTM!

clang/unittests/Interpreter/ExceptionTests/InterpreterExceptionTest.cpp

extern "C" int throw_exception() {
   ThrowerAnError("To be caught in binary");
   return 0;
 }
     )";
   std::unique_ptr<Interpreter> Interp = createInterpreter();
   // FIXME: Re-enable the excluded target triples.
   const clang::CompilerInstance *CI = Interp->getCompilerInstance();
   const llvm::Triple &Triple = CI->getASTContext().getTargetInfo().getTriple();
-  // FIXME: PPC fails due to `Symbols not found: [DW.ref.__gxx_personality_v0]`
-  // The current understanding is that the JIT should emit this symbol if it was
-  // not (eg. the way passing clang -fPIC does it).
-  if (Triple.isPPC())
-    return;
 
   // FIXME: ARM fails due to `Not implemented relocation type!`
   if (Triple.isARM())
     return;
 
   // FIXME: libunwind on darwin is broken, see PR49692.
   if (Triple.isOSDarwin() && (Triple.getArch() == llvm::Triple::aarch64 ||
                               Triple.getArch() == llvm::Triple::aarch64_32))

llvm/lib/ExecutionEngine/Orc/LLJIT.cpp

LLJIT::createObjectLinkingLayer(LLJITBuilderState &S, ExecutionSession &ES) {
   auto Layer =
       std::make_unique<RTDyldObjectLinkingLayer>(ES, std::move(GetMemMgr));
 
   if (S.JTMB->getTargetTriple().isOSBinFormatCOFF()) {
     Layer->setOverrideObjectFlagsWithResponsibilityFlags(true);
     Layer->setAutoClaimResponsibilityForObjectSymbols(true);
   }
 
+  if (S.JTMB->getTargetTriple().isOSBinFormatELF() &&
+      (S.JTMB->getTargetTriple().getArch() == Triple::ArchType::ppc64 ||
+       S.JTMB->getTargetTriple().getArch() == Triple::ArchType::ppc64le))
+    Layer->setAutoClaimResponsibilityForObjectSymbols(true);
+
   // FIXME: Explicit conversion to std::unique_ptr<ObjectLayer> added to silence
   //        errors from some GCC / libstdc++ bots. Remove this conversion (i.e.
   //        just return ObjLinkingLayer) once those bots are upgraded.
   return std::unique_ptr<ObjectLayer>(std::move(Layer));
 }
 
 Expected<std::unique_ptr<IRCompileLayer::IRCompiler>>
 LLJIT::createCompileFunction(LLJITBuilderState &S,

llvm/lib/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.cpp

if (!Obj) {
     R->failMaterialization();
     return;
   }
 
   // Collect the internal symbols from the object file: We will need to
   // filter these later.
   auto InternalSymbols = std::make_shared<std::set<StringRef>>();
   {
+    SymbolFlagsMap ExtraSymbolsToClaim;
     for (auto &Sym : (*Obj)->symbols()) {
 
       // Skip file symbols.
       if (auto SymType = Sym.getType()) {
         if (*SymType == object::SymbolRef::ST_File)
           continue;
       } else {
         ES.reportError(SymType.takeError());
         R->failMaterialization();
         return;
       }
 
       Expected<uint32_t> SymFlagsOrErr = Sym.getFlags();
       if (!SymFlagsOrErr) {
         // TODO: Test this error.
         ES.reportError(SymFlagsOrErr.takeError());
         R->failMaterialization();
         return;
       }
 
+      // Try to claim responsibility of weak symbols
+      // if AutoClaimObjectSymbols flag is set.
+      if (AutoClaimObjectSymbols &&
+          (*SymFlagsOrErr & object::BasicSymbolRef::SF_Weak)) {
+        auto SymName = Sym.getName();
+        if (!SymName) {
+          ES.reportError(SymName.takeError());
+          R->failMaterialization();
+          return;
+        }
+
+        // Already included in responsibility set, skip it
+        SymbolStringPtr SymbolName = ES.intern(*SymName);
+        if (R->getSymbols().count(SymbolName))
+          continue;
+
+        auto SymFlags = JITSymbolFlags::fromObjectSymbol(Sym);
+        if (!SymFlags) {
+          ES.reportError(SymFlags.takeError());
+          R->failMaterialization();
+          return;
+        }
+
+        ExtraSymbolsToClaim[SymbolName] = *SymFlags;
+        continue;
+      }
+
       // Don't include symbols that aren't global.
       if (!(*SymFlagsOrErr & object::BasicSymbolRef::SF_Global)) {
         if (auto SymName = Sym.getName())
           InternalSymbols->insert(*SymName);
         else {
           ES.reportError(SymName.takeError());
           R->failMaterialization();
           return;
         }
       }
     }
+
+    if (!ExtraSymbolsToClaim.empty()) {
+      if (auto Err = R->defineMaterializing(ExtraSymbolsToClaim)) {
+        ES.reportError(std::move(Err));
+        R->failMaterialization();
+      }
+    }
   }
 
   auto MemMgr = GetMemoryManager();
   auto &MemMgrRef = *MemMgr;
 
   // Switch to shared ownership of MR so that it can be captured by both
   // lambdas below.
   std::shared_ptr<MaterializationResponsibility> SharedR(std::move(R));