Page MenuHomePhabricator

[ORC] Add Platform and runtime support for ELF-based platforms
ClosedPublic

Authored by housel on Aug 14 2021, 3:30 PM.

Details

Summary

This change adds support to ORCv2 and the Orc runtime library for static initializers, C++ static destructors, and exception handler registration for ELF-based platforms, at present Linux and FreeBSD on x86_64. Support for thread-local storage is still incomplete.

Diff Detail

Event Timeline

housel created this revision.Aug 14 2021, 3:30 PM
housel requested review of this revision.Aug 14 2021, 3:30 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 14 2021, 3:30 PM
Herald added subscribers: llvm-commits, Restricted Project. · View Herald Transcript
housel updated this revision to Diff 366472.Aug 14 2021, 8:56 PM

Corrected file header comments

This looks great. Thanks Peter!

I've tested this out in a Linux container and verified that it correctly handles static constructors and destructors in both in-process and out-of-process mode. This is really exciting -- until now only Darwin users have been able to use these features. :)

Please see inline comments -- there are some vestigial TLV functions / members in the ORC runtime, which I think could be removed relatively easily. There are also some in the platform, but they would be tricker to take out -- I would be fine with them staying in in the initial commit.

Side note: This implementation replicates some of MachOPlatform's current warts (e.g. fuzzy operation ordering, JIT-side tracking of init data). This isn't a problem at all, we'll just need to be careful that any fixes made to MachOPlatform are also applied to ELFNix and vice-versa.

I know that you mentioned on discord that you do not have commit access: Would you mind if I removed the walkEHFrame and TLV functions from the runtime, then committed on your behalf?

compiler-rt/lib/orc/elfnix_platform.cpp
39–59

This is kind-of-sort-of out of place, but the situation is complex:

On Darwin the default unwinder is LLVM libunwind, and LLVM libunwind's __register_frame and __deregister_frame functions expect to be called on individual FDEs.

On Linux the default unwinder is libgcc_s, and libgcc_s's __register_frame and __deregister_frame functions expect to be called on whole eh-frame sections.

In LLVM at the moment we just assume that Darwin implies LLVM libunwind, and anything else implies libgcc_s, but this has also caused problems when people try to use LLVM libunwind on other platforms. See http://llvm.org/PR44074.

I think the best thing to do for now is to stay consistent with the current not-quite-right behavior of LLVM, and assume that ELFNix implies libgcc_s for unwinding: the walkEHFrameSection utility should be removed, and registerObjectSections and deregisterObjectSections should call __register_frame and __deregister_frame directly on the section ranges.

Separate from this review, we can revisit this as part of http://llvm.org/PR44074 -- I will update that bug with notes and cite this review.

164–165

These are unused and can be removed for now.

191–195

This can be removed and re-introduced in a future TLV patch.

275–286

I don't think this is called yet, so the whole function can be removed for now. It can be re-added in future a ELFNix TLV support patch.

327–339

This function is only used to put data into currently unused members, so it can be removed for now (and re-added in a future TLV support patch).

ELFNix is not used elsewhere. You may just call them ELF platforms.

llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
87

Use the section type SHT_INIT_ARRAY

ELFNix is not used elsewhere. You may just call them ELF platforms.

The ELFNix name was my idea. Platform support, at least as it's currently designed, is bound to both the object format and the OS (or Posix, in this case). I haven't come up with a way to neatly disentangle those two things yet, and it may not be possible to do so. MachOPlatform should really be MachODarwinPlatform -- I may rename that soon. (They're so synonymous that the full name didn't occur to me when I was writing it).

I know that you mentioned on discord that you do not have commit access: Would you mind if I removed the walkEHFrame and TLV functions from the runtime, then committed on your behalf?

Sure, makes sense.

This revision was not accepted when it landed; it landed in state Needs Review.Aug 17 2021, 10:01 PM
This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
dyung added a subscriber: dyung.Aug 18 2021, 3:31 AM

Hi, two of the tests added in this commit seem to be failing on our internal linux build bot:

FAIL: ORC-x86_64-linux :: TestCases/Linux/x86-64/trivial-cxa-atexit.S (77868 of 84146)
******************** TEST 'ORC-x86_64-linux :: TestCases/Linux/x86-64/trivial-cxa-atexit.S' FAILED ********************
Script:
--
: 'RUN: at line 3';      /home/dyung/src/upstream/build/./bin/clang   -m64  -c -o /home/dyung/src/upstream/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-cxa-atexit.S.tmp /home/dyung/src/upstream/llvm_clean_git/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-cxa-atexit.S
: 'RUN: at line 4';   /home/dyung/src/upstream/build/./bin/llvm-jitlink -orc-runtime=/home/dyung/src/upstream/build/./lib/clang/14.0.0/lib/linux/libclang_rt.orc-x86_64.a /home/dyung/src/upstream/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-cxa-atexit.S.tmp
--
Exit Code: 1

Command Output (stderr):
--
JIT session error: Unrecognized symbol binding 10 for _ZN8__orc_rt11RTTIExtendsINS_11StringErrorENS_13ErrorInfoBaseEE2IDE
/home/dyung/src/upstream/build/./bin/llvm-jitlink: Failed to materialize symbols: { (main, { _ZN8__orc_rt10make_errorINS_11StringErrorEJRA51_KcEEENS_5ErrorEDpOT0_, _ZTSN8__orc_rt11RTTIExtendsINS_11StringErrorENS_13ErrorInfoBaseEEE, _ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_St6vectorIN8__orc_rt20ExecutorAddressRangeESaISA_EEESaISD_ENSt8__detail10_Select1stESt8equal_toIS5_ESt4hashIS5_ENSF_18_Mod_range_hashingENSF_20_Default_ranged_hashENSF_20_Prime_rehash_policyENSF_17_Hashtable_traitsILb1ELb0ELb1EEEE13_M_rehash_auxEmSt17integral_constantIbLb1EE, __orc_rt_elfnix_jit_dlclose, _ZTIN8__orc_rt13ErrorInfoBaseE, __orc_rt_elfnix_jit_dlsym, _ZN8__orc_rt6detail38serializeViaSPSToWrapperFunctionResultINS_10SPSArgListIJNS_18SPSExecutorAddressENS_11SPSSequenceIcEEEEEJNS_15ExecutorAddressENS_11string_viewEEEENS_8ExpectedINS_21WrapperFunctionResultEEEDpRKT0_, __orc_rt_elfnix_deregister_object_sections, _ZN8__orc_rt8ExpectedIPvED1Ev, _ZN8__orc_rt10make_errorINS_11StringErrorEJNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEEENS_5ErrorEDpOT0_, _ZN8__orc_rt8ExpectedIPvED2Ev, _ZTIN8__orc_rt11RTTIExtendsINS_11StringErrorENS_13ErrorInfoBaseEEE, _ZTIN8__orc_rt11RTTIExtendsINS_13ErrorInfoBaseENS_8RTTIRootEEE, _ZTSN8__orc_rt11RTTIExtendsINS_13ErrorInfoBaseENS_8RTTIRootEEE, _ZN8__orc_rt8ExpectedINS_15ExecutorAddressEED2Ev, _ZN8__orc_rt8ExpectedISt6vectorINS_6elfnix26ELFNixJITDylibInitializersESaIS3_EEED2Ev, _ZTSN8__orc_rt13ErrorInfoBaseE, _ZN8__orc_rt6detail38serializeViaSPSToWrapperFunctionResultINS_10SPSArgListIJNS_8SPSErrorEEEEJNS0_20SPSSerializableErrorEEEENS_8ExpectedINS_21WrapperFunctionResultEEEDpRKT0_, _ZN8__orc_rt8ExpectedINS_15ExecutorAddressEED1Ev, _ZN8__orc_rt11RTTIExtendsINS_11StringErrorENS_13ErrorInfoBaseEE2IDE, _ZN8__orc_rt8ExpectedINS_21WrapperFunctionResultEED1Ev, _ZN8__orc_rt10make_errorINS_11StringErrorEJRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEEENS_5ErrorEDpOT0_, _ZN8__orc_rt10make_errorINS_11StringErrorEJRA44_KcEEENS_5ErrorEDpOT0_, __orc_rt_elfnix_jit_dlopen, _ZN8__orc_rt8ExpectedINS_21WrapperFunctionResultEED2Ev, __orc_rt_elfnix_symbol_lookup_tag, _ZN8__orc_rt15WrapperFunctionIFNS_11SPSExpectedINS_11SPSSequenceINS_8SPSTupleIJNS2_IcEENS_18SPSExecutorAddressENS2_INS3_IJS4_NS2_INS3_IJS5_S5_EEEEEEEEEEEEEEEEES4_EE4callINS_8ExpectedISt6vectorINS_6elfnix26ELFNixJITDylibInitializersESaISJ_EEEEJNS_11string_viewEEEENS_5ErrorEPKvRT_DpRKT0_, __orc_rt_elfnix_get_deinitializers_tag, __orc_rt_elfnix_jit_dlerror, _ZN8__orc_rt6detail38serializeViaSPSToWrapperFunctionResultINS_10SPSArgListIJNS_11SPSSequenceIcEEEEEJNS_11string_viewEEEENS_8ExpectedINS_21WrapperFunctionResultEEEDpRKT0_, __orc_rt_elfnix_platform_shutdown, _ZNSt8__detail9_Map_baseINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS6_PvESaISA_ENS_10_Select1stESt8equal_toIS6_ESt4hashIS6_ENS_18_Mod_range_hashingENS_20_Default_ranged_hashENS_20_Prime_rehash_policyENS_17_Hashtable_traitsILb1ELb0ELb1EEELb1EEixERS8_, _ZNSt6vectorIN8__orc_rt20ExecutorAddressRangeESaIS1_EE17_M_realloc_insertIJS1_EEEvN9__gnu_cxx17__normal_iteratorIPS1_S3_EEDpOT_, __orc_rt_elfnix_cxa_finalize, __orc_rt_elfnix_cxa_atexit, _ZN8__orc_rt22SPSSerializationTraitsINS_11SPSSequenceINS_8SPSTupleIJNS1_IcEENS1_INS2_IJNS_18SPSExecutorAddressES4_EEEEEEEEEESt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorINS_20ExecutorAddressRangeESaISH_EESt4hashISF_ESt8equal_toISF_ESaISt4pairIKSF_SJ_EEEvE11deserializeERNS_14SPSInputBufferERSS_, _ZNK8__orc_rt11StringError8toStringB5cxx11Ev, _ZNSt6vectorIN8__orc_rt6elfnix26ELFNixJITDylibInitializersESaIS2_EE17_M_realloc_insertIJS2_EEEvN9__gnu_cxx17__normal_iteratorIPS2_S4_EEDpOT_, _ZN8__orc_rt10make_errorINS_11StringErrorEJRA34_KcEEENS_5ErrorEDpOT0_, __orc_rt_elfnix_run_program, _ZN8__orc_rt6detail18ResultDeserializerINS_11SPSExpectedINS_11SPSSequenceINS_8SPSTupleIJNS3_IcEENS_18SPSExecutorAddressENS3_INS4_IJS5_NS3_INS4_IJS6_S6_EEEEEEEEEEEEEEEEENS_8ExpectedISt6vectorINS_6elfnix26ELFNixJITDylibInitializersESaISH_EEEEE11deserializeERSK_PKcm, _ZN8__orc_rt10make_errorINS_11StringErrorEJRA30_KcEEENS_5ErrorEDpOT0_, _ZNK8__orc_rt11RTTIExtendsINS_11StringErrorENS_13ErrorInfoBaseEE3isAEPKv, _ZN8__orc_rt8ExpectedISt6vectorINS_6elfnix26ELFNixJITDylibInitializersESaIS3_EEED1Ev, _ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_St6vectorIN8__orc_rt20ExecutorAddressRangeESaISA_EEESaISD_ENSt8__detail10_Select1stESt8equal_toIS5_ESt4hashIS5_ENSF_18_Mod_range_hashingENSF_20_Default_ranged_hashENSF_20_Prime_rehash_policyENSF_17_Hashtable_traitsILb1ELb0ELb1EEEE10_M_emplaceIJS6_IS5_SC_EEEES6_INSF_14_Node_iteratorISD_Lb0ELb1EEEbESt17integral_constantIbLb1EEDpOT_, _ZN8__orc_rt11RTTIExtendsINS_13ErrorInfoBaseENS_8RTTIRootEE2IDE, _ZNK8__orc_rt11RTTIExtendsINS_11StringErrorENS_13ErrorInfoBaseEE14dynamicClassIDEv, _ZN8__orc_rt10make_errorINS_11StringErrorEJRA65_KcEEENS_5ErrorEDpOT0_, _ZN8__orc_rt11StringErrorD2Ev, __orc_rt_elfnix_register_object_sections, _ZTVN8__orc_rt11StringErrorE, _ZTIN8__orc_rt11StringErrorE, __orc_rt_elfnix_get_initializers_tag, _ZN8__orc_rt11StringErrorD1Ev, _ZTSN8__orc_rt11StringErrorE, _ZNKSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_PvESaIS9_ENSt8__detail10_Select1stESt8equal_toIS5_ESt4hashIS5_ENSB_18_Mod_range_hashingENSB_20_Default_ranged_hashENSB_20_Prime_rehash_policyENSB_17_Hashtable_traitsILb1ELb0ELb1EEEE5countERS7_, _ZN8__orc_rt11StringErrorD0Ev, __orc_rt_elfnix_platform_bootstrap }) }

The other failing test is TestCases/Linux/x86-64/trivial-static-initializer.S which failed with a similar error. This is on Ubuntu 20.04 using gcc 9.3.0. Can you take a look?

Hi, two of the tests added in this commit seem to be failing on our internal linux build bot:

Hi Douglas. I think this is *probably* STB_GNU_UNIQUE, but haven't been able to reproduce this locally yet. Would you mind emailing me the -verbose output from
``
/home/dyung/src/upstream/build/./bin/clang -m64 -S -o /home/dyung/src/upstream/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-cxa-atexit.S /home/dyung/src/upstream/llvm_clean_git/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-cxa-atexit.S -v

I'll revert the patch for now.
dyung added a comment.Aug 18 2021, 3:46 AM

Hi, two of the tests added in this commit seem to be failing on our internal linux build bot:

Hi Douglas. I think this is *probably* STB_GNU_UNIQUE, but haven't been able to reproduce this locally yet. Would you mind emailing me the -verbose output from
``
/home/dyung/src/upstream/build/./bin/clang -m64 -S -o /home/dyung/src/upstream/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-cxa-atexit.S /home/dyung/src/upstream/llvm_clean_git/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-cxa-atexit.S -v

I'll revert the patch for now.

Thanks, I've emailed you the verbose output. Let me know if you didn't receive it.