Page MenuHomePhabricator

[ORC] Align weak definition handling in JITDylib::defineMaterializing() and JITDylib::defineImpl()
AbandonedPublic

Authored by sgraenitz on Feb 24 2020, 4:10 PM.

Details

Reviewers
lhames
Summary

In both functions handle all cases for symbol name collisions:

(1) The incoming symbol is weak: reject the incoming symbol definition
(2) The existing symbol is weak (and not materialized!): discard the existing symbol definition and insert the incoming one
(3) Both symbol definitions are strong: DuplicateDefinition error

This behavior would be very handy for ThinLtoJIT, because it (currently) uses weak definitions for fallback handlers. One example: let's say we have two modules, main.bc and rare.bc. The latter defines a function rarely() that is called from main.bc, so it will be resolved as a transitive dependency when linking main.bc even though it might never be called (or not even be reachable from our entry point!).

ThinLtoJIT will try to minimize the immediate/synchronous workload and bet on discovery to identify and emit all necessary code asynchronously. In many cases this may work, but it still needs a fallback in case rarely() gets called and discovery didn't manage to provide rare.bc in time. For this case it emits a synthetic call-through stub that does the work (load and emit the module). The symbol it defines for rarely() in the shadow dylib would then simply be weak, so that discovery can override it with the actual definition from rare.bc once it's ready.

Event Timeline

sgraenitz created this revision.Feb 24 2020, 4:10 PM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 24 2020, 4:10 PM
lhames added a comment.Mar 3 2020, 5:23 PM

ThinLtoJIT will try to minimize the immediate/synchronous workload and bet on discovery to identify and emit all necessary code asynchronously. In many cases this may work, but it still needs a fallback in case rarely() gets called and discovery didn't manage to provide rare.bc in time. For this case it emits a synthetic call-through stub that does the work (load and emit the module). The symbol it defines for rarely() in the shadow dylib would then simply be weak, so that discovery can override it with the actual definition from rare.bc once it's ready.

Is the idea here that the call-through stub is emitted as weak, and then replaced by discovery with a strong definition if one is found in time? Or have I misunderstood?

Is the idea here that the call-through stub is emitted as weak, and then replaced by discovery with a strong definition if one is found in time? Or have I misunderstood?

Yes that's the idea. (Though the actual definition found by discovery might be weak itself -- but I don't handle this yet.) I think the cases where I ran into this are actually race conditions in my code, where multiple threads compete for symbol table access. While I will have to revisit this part, I wondered whether it may be worth having an aligned mechanism for both cases. Something like the (1), (2), (3) described above?

sgraenitz abandoned this revision.Aug 14 2020, 7:58 AM