[Orc][JITLink] Add a shared memory based implementation of MemoryMapper

[argentite]

Summary

This is an implementation of orc::MemoryMapper that maps shared memory
pages in both executor and controller process and writes directly to
them avoiding transferring content over EPC.
All allocations are properly deinitialized automatically on the
executor side at shutdown by the ExecutorSharedMemoryMapperService.

[argentite]

Windows support and some other fixes

[argentite]

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

26–49

This function is copied from lib/Support/Windows/Memory.inc. We probably should expose it from there instead.

[sgraenitz]

Thanks for posting your patch! I will continue reviewing, but here is a first detail I found..

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

229

I had to cast this to uint64_t explicitly in order to fix a SPS encoding error. It appears the issue is that on my system size_t is unsigned long and not unsigned long long.

[argentite]

Added explicit cast from size_t to uint64_t

[lhames]

This looks great. Thanks @argentite!

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

362

The call to release could fail -- we'll want to figure out the right error plumbing for this in the long run. Could you add a // FIXME for now?

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

157

Are there any other guards to prevent ExecutorSharedMemoryMapperService from being instantiated / used on unsupported platforms? If not, I think this should return an llvm::Error for now.

llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp

59–64

Looks like these could be moved into the scope below.

60–61

There's no explicit test that the lambdas are executed, but I think the use of this promise/future pair in the innermost lambda is sufficient -- the test will block unless all lambdas are run. That's probably worth a comment to point out though.

[argentite]

Replaced some llvm_unreachable() with llvm::Error

[argentite]

Skip the test on unimplemented operating systems

[amyk]

Hi there!

I just wanted to give a heads up that it looks like this patch is causing failures on some of the PPC bots that build with shared libraries:
https://lab.llvm.org/buildbot/#/builders/57
https://lab.llvm.org/buildbot/#/builders/121

The error looks like this in https://lab.llvm.org/buildbot/#/builders/121/builds/21343/steps/4/logs/stdio:

[124/1619] Linking CXX shared library lib/libLLVMOrcTargetProcess.so.15git
FAILED: lib/libLLVMOrcTargetProcess.so.15git 
: && /usr/lib64/ccache/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/buildbots/ppc64le-clang-multistage-test/clang-ppc64le-multistage/stage1/./lib  -Wl,--gc-sections -shared -Wl,-soname,libLLVMOrcTargetProcess.so.15git -o lib/libLLVMOrcTargetProcess.so.15git lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/ExecutorSharedMemoryMapperService.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/JITLoaderGDB.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/OrcRTBootstrap.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/RegisterEHFrames.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/SimpleExecutorDylibManager.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/SimpleExecutorMemoryManager.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/SimpleRemoteEPCServer.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/TargetExecutionUtils.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  -lpthread  lib/libLLVMOrcShared.so.15git  lib/libLLVMSupport.so.15git  -Wl,-rpath-link,/home/buildbots/ppc64le-clang-multistage-test/clang-ppc64le-multistage/stage1/lib && :
lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/ExecutorSharedMemoryMapperService.cpp.o: In function `llvm::orc::rt_bootstrap::ExecutorSharedMemoryMapperService::reserve[abi:cxx11](unsigned long)':
ExecutorSharedMemoryMapperService.cpp:(.text._ZN4llvm3orc12rt_bootstrap33ExecutorSharedMemoryMapperService7reserveB5cxx11Em+0x398): undefined reference to `shm_open'
collect2: error: ld returned 1 exit status

Are you able to look into this?

Thanks.

[argentite]

Hi! I am not very familiar with powerpc but it seems like POSIX shared memory APIs should be available.
From this page I suspect shm_ family of functions could be in librt instead of libc.

[abinavpp]

In D128544#3650815, @argentite wrote:

Hi! I am not very familiar with powerpc but it seems like POSIX shared memory APIs should be available.
From this page I suspect shm_ family of functions could be in librt instead of libc.

I don't think this is specific to PPC, shm_* APIs are in librt right? I can see "Link with -lrt." in the synopsis of https://man7.org/linux/man-pages/man3/shm_open.3.html. So I guess doing something like:

diff --git a/llvm/lib/ExecutionEngine/CMakeLists.txt b/llvm/lib/ExecutionEngine/CMakeLists.txt
index a5607872c706..cdb7b3e622c3 100644
--- a/llvm/lib/ExecutionEngine/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/CMakeLists.txt
@@ -24,7 +24,7 @@ add_llvm_component_library(LLVMExecutionEngine
   )
 
 if(BUILD_SHARED_LIBS)
-  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld)
+  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld rt)
 endif()
 
 add_subdirectory(Interpreter)
diff --git a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
index 415876abd65f..9a2709a4d6af 100644
--- a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
@@ -12,7 +12,7 @@ add_llvm_component_library(LLVMOrcTargetProcess
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/Orc
 
   LINK_LIBS
-  ${LLVM_PTHREAD_LIB}
+  ${LLVM_PTHREAD_LIB} rt
 
   LINK_COMPONENTS
   OrcShared

should get the shared-lib build fixed in Linux, but I'm not sure about other platforms.

[c-rhodes]

I've reverted this in 3e9cc543f223 since I'm hitting the same error as @amyk compiling with shared libs on AArch64 Linux. I had to revert a fix (46b1a7c5f9e6) from @mstorsjo for mingw as well so you probably want to pick that up when re-landing.

[thakis]

In D128544#3650873, @abinavpp wrote:

In D128544#3650815, @argentite wrote:

Hi! I am not very familiar with powerpc but it seems like POSIX shared memory APIs should be available.
From this page I suspect shm_ family of functions could be in librt instead of libc.

I don't think this is specific to PPC, shm_* APIs are in librt right? I can see "Link with -lrt." in the synopsis of https://man7.org/linux/man-pages/man3/shm_open.3.html. So I guess doing something like:

diff --git a/llvm/lib/ExecutionEngine/CMakeLists.txt b/llvm/lib/ExecutionEngine/CMakeLists.txt
index a5607872c706..cdb7b3e622c3 100644
--- a/llvm/lib/ExecutionEngine/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/CMakeLists.txt
@@ -24,7 +24,7 @@ add_llvm_component_library(LLVMExecutionEngine
   )
 
 if(BUILD_SHARED_LIBS)
-  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld)
+  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld rt)
 endif()
 
 add_subdirectory(Interpreter)
diff --git a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
index 415876abd65f..9a2709a4d6af 100644
--- a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
@@ -12,7 +12,7 @@ add_llvm_component_library(LLVMOrcTargetProcess
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/Orc
 
   LINK_LIBS
-  ${LLVM_PTHREAD_LIB}
+  ${LLVM_PTHREAD_LIB} rt
 
   LINK_COMPONENTS
   OrcShared

should get the shared-lib build fixed in Linux, but I'm not sure about other platforms.

This looks right to me, but it shouldn't be done on windows and apple platforms i think. (Not sure about the bsds.)

Maybe look for what we do for HAVE_LIBRT in llvm/cmake/config-ix.cmake and set a var like LLVM_RT_LIB there and use that here (similar to LLVM_PTHREAD_LIB).

[argentite]

Link with librt if available

I think this should work everywhere.

It seems like some platforms have shm_ functions in libc and librt acts like a placeholder.

[c-rhodes]

In D128544#3651546, @argentite wrote:

Link with librt if available

I think this should work everywhere.

It seems like some platforms have shm_ functions in libc and librt acts like a placeholder.

This compiles for me now, cheers

[lhames]

LGTM otherwise -- thanks for chasing up the fix @argentite.

llvm/lib/ExecutionEngine/Orc/CMakeLists.txt

3

As @thakis suggested I think this should be LLVM_RT_LIB (for consistency with LLVM_PTHREAD_LIB).

[lhames]

As @thakis suggested I think this should be LLVM_RT_LIB (for consistency with LLVM_PTHREAD_LIB).

Re-committed on @argentite's behalf. I ended up leaving this as-is: it looks like libSupport currently does something similar with a local CMake variable for rt. We can introduce LLVM_RT_LIB in a follow-up patch if we decide it's worth it.

[hctim]

Looks like this broke the UBSan buildbot:

https://lab.llvm.org/buildbot/#/builders/85/builds/9186

Details on how to reproduce sanitizer buildbots can be found at https://github.com/google/sanitizers/wiki/SanitizerBotReproduceBuild, but hopefully the error from the bot should be enough to identify the issue.

-- Testing: 67188 tests, 80 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90
FAIL: LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/84/128 (61930 of 67188)
******************** TEST 'LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/84/128' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm_build_ubsan/unittests/ExecutionEngine/Orc/./OrcJITTests-LLVM-Unit-4179923-84-128.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=128 GTEST_SHARD_INDEX=84 /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm_build_ubsan/unittests/ExecutionEngine/Orc/./OrcJITTests
--
Note: This is test shard 85 of 128.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from SharedMemoryMapperTest
[ RUN      ] SharedMemoryMapperTest.MemReserveInitializeDeinitializeRelease
/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:61:20: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/lib/gcc-cross/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include/string.h:44:28: note: nonnull attribute specified here
    #0 0x5567ac3b1159 in llvm::orc::shared::SPSOutputBuffer::write(char const*, unsigned long) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:61:5
    #1 0x5567ac74da31 in serialize /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:352:15
    #2 0x5567ac74da31 in serialize<llvm::ArrayRef<char> > /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #3 0x5567ac74da31 in bool llvm::orc::shared::SPSArgList<unsigned long, llvm::orc::shared::SPSSequence<char>>::serialize<unsigned long, llvm::ArrayRef<char>>(llvm::orc::shared::SPSOutputBuffer&, unsigned long const&, llvm::ArrayRef<char> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #4 0x5567ac74d9a4 in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>::serialize<llvm::orc::ExecutorAddr, unsigned long, llvm::ArrayRef<char>>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::ExecutorAddr const&, unsigned long const&, llvm::ArrayRef<char> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #5 0x5567ac74d95d in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>::serialize<llvm::orc::tpctypes::WireProtectionFlags, llvm::orc::ExecutorAddr, unsigned long, llvm::ArrayRef<char>>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::tpctypes::WireProtectionFlags const&, llvm::orc::ExecutorAddr const&, unsigned long const&, llvm::ArrayRef<char> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #6 0x5567ac74d8a8 in serialize<llvm::orc::tpctypes::SegFinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #7 0x5567ac74d8a8 in llvm::orc::shared::SPSSerializationTraits<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, std::__1::vector<llvm::orc::tpctypes::SegFinalizeRequest, std::__1::allocator<llvm::orc::tpctypes::SegFinalizeRequest>>, void>::serialize(llvm::orc::shared::SPSOutputBuffer&, std::__1::vector<llvm::orc::tpctypes::SegFinalizeRequest, std::__1::allocator<llvm::orc::tpctypes::SegFinalizeRequest>> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:385:12
    #8 0x5567ac74d65f in serialize<std::__1::vector<llvm::orc::tpctypes::SegFinalizeRequest, std::__1::allocator<llvm::orc::tpctypes::SegFinalizeRequest> >, std::__1::vector<llvm::orc::shared::AllocActionCallPair, std::__1::allocator<llvm::orc::shared::AllocActionCallPair> > > /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #9 0x5567ac74d65f in serialize /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h:199:12
    #10 0x5567ac74d65f in serialize<llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #11 0x5567ac74d65f in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>>>>>::serialize<llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::ExecutorAddr const&, llvm::orc::tpctypes::FinalizeRequest const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #12 0x5567ac76cbc4 in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>>>>>::serialize<llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::ExecutorAddr const&, llvm::orc::ExecutorAddr const&, llvm::orc::tpctypes::FinalizeRequest const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #13 0x5567ac76c8cb in llvm::orc::shared::WrapperFunctionResult llvm::orc::shared::detail::serializeViaSPSToWrapperFunctionResult<llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>>>>>, llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest>(llvm::orc::ExecutorAddr const&, llvm::orc::ExecutorAddr const&, llvm::orc::tpctypes::FinalizeRequest const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h:176:8
    #14 0x5567ac7685c4 in callAsync<(lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:317:9), (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:281:7), llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h:484:9
    #15 0x5567ac7685c4 in callSPSWrapperAsync<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSExecutorAddr> (llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char> > >, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> >, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> > > > >), llvm::orc::ExecutorProcessControl::RunAsTask, (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:281:7), llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:316:5
    #16 0x5567ac7685c4 in callSPSWrapperAsync<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSExecutorAddr> (llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char> > >, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> >, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> > > > >), (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:281:7), llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:331:5
    #17 0x5567ac7685c4 in llvm::orc::SharedMemoryMapper::initialize(llvm::orc::MemoryMapper::AllocInfo&, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddr>)>) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:278:7
    #18 0x5567ac3e1c28 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp:100:15
    #19 0x5567ac3e1c28 in void llvm::detail::UniqueFunctionBase<void, llvm::Expected<llvm::orc::ExecutorAddrRange>>::CallImpl<SharedMemoryMapperTest_MemReserveInitializeDeinitializeRelease_Test::TestBody()::$_1>(void*, llvm::Expected<llvm::orc::ExecutorAddrRange>&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ADT/FunctionExtras.h:222:12
    #20 0x5567ac76b10a in llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2::operator()(llvm::Error, llvm::Expected<std::__1::pair<llvm::orc::ExecutorAddr, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:231:9
    #21 0x5567ac76be35 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h
    #22 0x5567ac76be35 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:92:23
    #23 0x5567ac76be35 in llvm::orc::GenericNamedTaskImpl<llvm::orc::ExecutorProcessControl::IncomingWFRHandler llvm::orc::ExecutorProcessControl::RunAsTask::operator()<void llvm::orc::shared::WrapperFunction<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>> (llvm::orc::shared::SPSExecutorAddr, unsigned long)>::callAsync<void llvm::orc::ExecutorProcessControl::callSPSWrapperAsync<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>> (llvm::orc::shared::SPSExecutorAddr, unsigned long), llvm::orc::ExecutorProcessControl::RunAsTask, llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2, llvm::orc::ExecutorAddr, unsigned long>(llvm::orc::ExecutorProcessControl::RunAsTask&&, llvm::orc::ExecutorAddr, llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2&&, llvm::orc::ExecutorAddr const&, unsigned long const&)::'lambda'(auto&&, char const*, unsigned long), llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2, llvm::orc::ExecutorAddr, unsigned long>(auto&&, llvm::orc::ExecutorProcessControl::RunAsTask&&, llvm::orc::ExecutorAddr const&, unsigned long const&)::'lambda'(llvm::orc::shared::WrapperFunctionResult)>(auto&&)::'lambda'(llvm::orc::shared::WrapperFunctionResult)::operator()(llvm::orc::shared::WrapperFunctionResult)::'lambda'()>::run() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/TaskDispatch.h:68:25
    #24 0x5567ac79f190 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/TaskDispatch.cpp:33:8
    #25 0x5567ac79f190 in __invoke<(lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/TaskDispatch.cpp:32:15)> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/libcxx_build_ubsan/include/c++/v1/__functional/invoke.h:403:23
    #26 0x5567ac79f190 in __thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/TaskDispatch.cpp:32:15)> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/libcxx_build_ubsan/include/c++/v1/thread:284:5
    #27 0x5567ac79f190 in void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, llvm::orc::DynamicThreadPoolTaskDispatcher::dispatch(std::__1::unique_ptr<llvm::orc::Task, std::__1::default_delete<llvm::orc::Task>>)::$_0>>(void*) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/libcxx_build_ubsan/include/c++/v1/thread:295:5
    #28 0x7fc9f385eb42  (/lib/x86_64-linux-gnu/libc.so.6+0x94b42) (BuildId: 89c3cb85f9e55046776471fed05ec441581d1969)
    #29 0x7fc9f38f09ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff) (BuildId: 89c3cb85f9e55046776471fed05ec441581d1969)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:61:20 in

[argentite]

Use new SharedMemorySegFinalizeRequest instead of SegFinalizeRequest

The problem seems to be that there is a Content: ArrayRef<char> in SegFinalizeRequest. In SharedMemoryMapper it is set to (nullptr, 0) since we do not use it but the serialization code runs memcpy anyway with source nullptr and length 0. A new type SharedMemorySegFinalizeRequest without this field will make the intent clear as well.

[lhames]

In D128544#3659757, @argentite wrote:

Use new SharedMemorySegFinalizeRequest instead of SegFinalizeRequest

The problem seems to be that there is a Content: ArrayRef<char> in SegFinalizeRequest. In SharedMemoryMapper it is set to (nullptr, 0) since we do not use it but the serialization code runs memcpy anyway with source nullptr and length 0. A new type SharedMemorySegFinalizeRequest without this field will make the intent clear as well.

I fixed serialization of empty ArrayRef<char> in 67220c2ad72, but I like the idea of a custom request type for the SharedMemoryMapper too -- I'll keep that and re-re-land this.

[lhames]

Re-re-committed in https://reviews.llvm.org/rG1b1f1c778695442c2683813ec7d4b557bb279e26.

[RKSimon]

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

278

@argentite Static analysis is complaining that this second return is dead code - is it or should the return release() be inside the if() ?

[argentite]

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

278

Yup that release should be inside the if body but we also can't hold the mutex while calling release. I reordered them here: https://reviews.llvm.org/D131510 .

llvm/include/llvm/ExecutionEngine/Orc/MemoryMapper.h

private:
   };
   using ReservationMap = DenseMap<void *, Reservation>;
 
   std::mutex Mutex;
   ReservationMap Reservations;
   AllocationMap Allocations;
 };
 
+class SharedMemoryMapper final : public MemoryMapper {
+public:
+  struct SymbolAddrs {
+    ExecutorAddr Instance;
+    ExecutorAddr Reserve;
+    ExecutorAddr Initialize;
+    ExecutorAddr Deinitialize;
+    ExecutorAddr Release;
+  };
+
+  SharedMemoryMapper(ExecutorProcessControl &EPC, SymbolAddrs SAs)
+      : EPC(EPC), SAs(SAs) {}
+
+  void reserve(size_t NumBytes, OnReservedFunction OnReserved) override;
+
+  char *prepare(ExecutorAddr Addr, size_t ContentSize) override;
+
+  void initialize(AllocInfo &AI, OnInitializedFunction OnInitialized) override;
+
+  void deinitialize(ArrayRef<ExecutorAddr> Allocations,
+                    OnDeinitializedFunction OnDeInitialized) override;
+
+  void release(ArrayRef<ExecutorAddr> Reservations,
+               OnReleasedFunction OnRelease) override;
+
+  ~SharedMemoryMapper() override;
+
+private:
+  struct Reservation {
+    void *LocalAddr;
+    size_t Size;
+  };
+
+  ExecutorProcessControl &EPC;
+  SymbolAddrs SAs;
+
+  std::mutex Mutex;
+
+  std::map<ExecutorAddr, Reservation> Reservations;
+};
+
 } // namespace orc
 } // end namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_ORC_MEMORYMAPPER_H

llvm/include/llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h

 extern const char *SimpleExecutorDylibManagerOpenWrapperName;
 extern const char *SimpleExecutorDylibManagerLookupWrapperName;
 
 extern const char *SimpleExecutorMemoryManagerInstanceName;
 extern const char *SimpleExecutorMemoryManagerReserveWrapperName;
 extern const char *SimpleExecutorMemoryManagerFinalizeWrapperName;
 extern const char *SimpleExecutorMemoryManagerDeallocateWrapperName;
 
+extern const char *ExecutorSharedMemoryMapperServiceInstanceName;
+extern const char *ExecutorSharedMemoryMapperServiceReserveWrapperName;
+extern const char *ExecutorSharedMemoryMapperServiceInitializeWrapperName;
+extern const char *ExecutorSharedMemoryMapperServiceDeinitializeWrapperName;
+extern const char *ExecutorSharedMemoryMapperServiceReleaseWrapperName;
+
 extern const char *MemoryWriteUInt8sWrapperName;
 extern const char *MemoryWriteUInt16sWrapperName;
 extern const char *MemoryWriteUInt32sWrapperName;
 extern const char *MemoryWriteUInt64sWrapperName;
 extern const char *MemoryWriteBuffersWrapperName;
 
 extern const char *RegisterEHFrameSectionWrapperName;
 extern const char *DeregisterEHFrameSectionWrapperName;

 using SPSSimpleExecutorMemoryManagerReserveSignature =
     shared::SPSExpected<shared::SPSExecutorAddr>(shared::SPSExecutorAddr,
                                                  uint64_t);
 using SPSSimpleExecutorMemoryManagerFinalizeSignature =
     shared::SPSError(shared::SPSExecutorAddr, shared::SPSFinalizeRequest);
 using SPSSimpleExecutorMemoryManagerDeallocateSignature = shared::SPSError(
     shared::SPSExecutorAddr, shared::SPSSequence<shared::SPSExecutorAddr>);
 
+// ExecutorSharedMemoryMapperService
+using SPSExecutorSharedMemoryMapperServiceReserveSignature =
+    shared::SPSExpected<
+        shared::SPSTuple<shared::SPSExecutorAddr, shared::SPSString>>(
+        shared::SPSExecutorAddr, uint64_t);
+using SPSExecutorSharedMemoryMapperServiceInitializeSignature =
+    shared::SPSExpected<shared::SPSExecutorAddr>(
+        shared::SPSExecutorAddr, shared::SPSExecutorAddr,
+        shared::SPSSharedMemoryFinalizeRequest);
+using SPSExecutorSharedMemoryMapperServiceDeinitializeSignature =
+    shared::SPSError(shared::SPSExecutorAddr,
+                     shared::SPSSequence<shared::SPSExecutorAddr>);
+using SPSExecutorSharedMemoryMapperServiceReleaseSignature = shared::SPSError(
+    shared::SPSExecutorAddr, shared::SPSSequence<shared::SPSExecutorAddr>);
+
 using SPSRunAsMainSignature = int64_t(shared::SPSExecutorAddr,
                                       shared::SPSSequence<shared::SPSString>);
 
 } // end namespace rt
 } // end namespace orc
 } // end namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_ORC_SHARED_ORCRTBRIDGE_H

llvm/include/llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h

struct SegFinalizeRequest {
   ArrayRef<char> Content;
 };
 
 struct FinalizeRequest {
   std::vector<SegFinalizeRequest> Segments;
   shared::AllocActions Actions;
 };
 
+struct SharedMemorySegFinalizeRequest {
+  WireProtectionFlags Prot;
+  ExecutorAddr Addr;
+  uint64_t Size;
+};
+
+struct SharedMemoryFinalizeRequest {
+  std::vector<SharedMemorySegFinalizeRequest> Segments;
+  shared::AllocActions Actions;
+};
+
 template <typename T> struct UIntWrite {
   UIntWrite() = default;
   UIntWrite(ExecutorAddr Addr, T Value) : Addr(Addr), Value(Value) {}
 
   ExecutorAddr Addr;
   T Value = 0;
 };
 

 
 using SPSSegFinalizeRequest =
     SPSTuple<SPSMemoryProtectionFlags, SPSExecutorAddr, uint64_t,
              SPSSequence<char>>;
 
 using SPSFinalizeRequest = SPSTuple<SPSSequence<SPSSegFinalizeRequest>,
                                     SPSSequence<SPSAllocActionCallPair>>;
 
+using SPSSharedMemorySegFinalizeRequest =
+    SPSTuple<SPSMemoryProtectionFlags, SPSExecutorAddr, uint64_t>;
+
+using SPSSharedMemoryFinalizeRequest =
+    SPSTuple<SPSSequence<SPSSharedMemorySegFinalizeRequest>,
+             SPSSequence<SPSAllocActionCallPair>>;
+
 template <typename T>
 using SPSMemoryAccessUIntWrite = SPSTuple<SPSExecutorAddr, T>;
 
 using SPSMemoryAccessUInt8Write = SPSMemoryAccessUIntWrite<uint8_t>;
 using SPSMemoryAccessUInt16Write = SPSMemoryAccessUIntWrite<uint16_t>;
 using SPSMemoryAccessUInt32Write = SPSMemoryAccessUIntWrite<uint32_t>;
 using SPSMemoryAccessUInt64Write = SPSMemoryAccessUIntWrite<uint64_t>;
 
static bool serialize(SPSOutputBuffer &OB,
     return FRAL::serialize(OB, FR.Segments, FR.Actions);
   }
 
   static bool deserialize(SPSInputBuffer &IB, tpctypes::FinalizeRequest &FR) {
     return FRAL::deserialize(IB, FR.Segments, FR.Actions);
   }
 };
 
+template <>
+class SPSSerializationTraits<SPSSharedMemorySegFinalizeRequest,
+                             tpctypes::SharedMemorySegFinalizeRequest> {
+  using SFRAL = SPSSharedMemorySegFinalizeRequest::AsArgList;
+
+public:
+  static size_t size(const tpctypes::SharedMemorySegFinalizeRequest &SFR) {
+    return SFRAL::size(SFR.Prot, SFR.Addr, SFR.Size);
+  }
+
+  static bool serialize(SPSOutputBuffer &OB,
+                        const tpctypes::SharedMemorySegFinalizeRequest &SFR) {
+    return SFRAL::serialize(OB, SFR.Prot, SFR.Addr, SFR.Size);
+  }
+
+  static bool deserialize(SPSInputBuffer &IB,
+                          tpctypes::SharedMemorySegFinalizeRequest &SFR) {
+    return SFRAL::deserialize(IB, SFR.Prot, SFR.Addr, SFR.Size);
+  }
+};
+
+template <>
+class SPSSerializationTraits<SPSSharedMemoryFinalizeRequest,
+                             tpctypes::SharedMemoryFinalizeRequest> {
+  using FRAL = SPSSharedMemoryFinalizeRequest::AsArgList;
+
+public:
+  static size_t size(const tpctypes::SharedMemoryFinalizeRequest &FR) {
+    return FRAL::size(FR.Segments, FR.Actions);
+  }
+
+  static bool serialize(SPSOutputBuffer &OB,
+                        const tpctypes::SharedMemoryFinalizeRequest &FR) {
+    return FRAL::serialize(OB, FR.Segments, FR.Actions);
+  }
+
+  static bool deserialize(SPSInputBuffer &IB,
+                          tpctypes::SharedMemoryFinalizeRequest &FR) {
+    return FRAL::deserialize(IB, FR.Segments, FR.Actions);
+  }
+};
+
 template <typename T>
 class SPSSerializationTraits<SPSMemoryAccessUIntWrite<T>,
                              tpctypes::UIntWrite<T>> {
 public:
   static size_t size(const tpctypes::UIntWrite<T> &W) {
     return SPSTuple<SPSExecutorAddr, T>::AsArgList::size(W.Addr, W.Value);
   }
 
public:
 
   static bool deserialize(SPSInputBuffer &IB, tpctypes::BufferWrite &W) {
     return SPSTuple<SPSExecutorAddr,
                     SPSSequence<char>>::AsArgList ::deserialize(IB, W.Addr,
                                                                 W.Buffer);
   }
 };
 
-
 } // end namespace shared
 } // end namespace orc
 } // end namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_ORC_SHARED_TARGETPROCESSCONTROLTYPES_H

llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.h

+//===----------- ExecutorSharedMemoryMapperService.h ------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_EXECUTORSHAREDMEMORYMAPPERSERVICE
+#define LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_EXECUTORSHAREDMEMORYMAPPERSERVICE
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h"
+#include "llvm/ExecutionEngine/Orc/TargetProcess/ExecutorBootstrapService.h"
+
+#include <atomic>
+#include <mutex>
+
+#if defined(_WIN32)
+#include <windows.h>
+#endif
+
+namespace llvm {
+namespace orc {
+namespace rt_bootstrap {
+
+class ExecutorSharedMemoryMapperService final
+    : public ExecutorBootstrapService {
+public:
+  ~ExecutorSharedMemoryMapperService(){};
+
+  Expected<std::pair<ExecutorAddr, std::string>> reserve(uint64_t Size);
+  Expected<ExecutorAddr> initialize(ExecutorAddr Reservation,
+                                    tpctypes::SharedMemoryFinalizeRequest &FR);
+
+  Error deinitialize(const std::vector<ExecutorAddr> &Bases);
+  Error release(const std::vector<ExecutorAddr> &Bases);
+
+  Error shutdown() override;
+  void addBootstrapSymbols(StringMap<ExecutorAddr> &M) override;
+
+private:
+  struct Allocation {
+    std::vector<shared::WrapperFunctionCall> DeinitializationActions;
+  };
+  using AllocationMap = DenseMap<ExecutorAddr, Allocation>;
+
+  struct Reservation {
+    size_t Size;
+    std::vector<ExecutorAddr> Allocations;
+#if defined(_WIN32)
+    HANDLE SharedMemoryFile;
+#endif
+  };
+  using ReservationMap = DenseMap<void *, Reservation>;
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  reserveWrapper(const char *ArgData, size_t ArgSize);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  initializeWrapper(const char *ArgData, size_t ArgSize);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  deinitializeWrapper(const char *ArgData, size_t ArgSize);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  releaseWrapper(const char *ArgData, size_t ArgSize);
+
+  std::atomic<int> SharedMemoryCount{0};
+  std::mutex Mutex;
+  ReservationMap Reservations;
+  AllocationMap Allocations;
+};
+
+} // namespace rt_bootstrap
+} // namespace orc
+} // namespace llvm
+#endif // LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_EXECUTORSHAREDMEMORYMAPPERSERVICE

llvm/lib/ExecutionEngine/Orc/CMakeLists.txt

+if( CMAKE_HOST_UNIX AND HAVE_LIBRT )
+  set(rt_lib rt)
+endif()
+
 add_llvm_component_library(LLVMOrcJIT
   CompileOnDemandLayer.cpp
   CompileUtils.cpp
   Core.cpp
   DebugObjectManagerPlugin.cpp
   DebuggerSupportPlugin.cpp
   DebugUtils.cpp
   EPCDynamicLibrarySearchGenerator.cpp
add_llvm_component_library(LLVMOrcJIT
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/Orc
 
   DEPENDS
   intrinsics_gen
 
   LINK_LIBS
   ${LLVM_PTHREAD_LIB}
+  ${rt_lib}
 
   LINK_COMPONENTS
   Core
   ExecutionEngine
   JITLink
   Object
   OrcShared
   OrcTargetProcess

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

 //===- MemoryMapper.cpp - Cross-process memory mapper ------------*- C++ -*-==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ExecutionEngine/Orc/MemoryMapper.h"
 
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+#include "llvm/Support/WindowsError.h"
+
+#if defined(LLVM_ON_UNIX)
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#elif defined(_WIN32)
+#include <windows.h>
+#endif
+
 namespace llvm {
 namespace orc {
 
 MemoryMapper::~MemoryMapper() {}
 
 void InProcessMemoryMapper::reserve(size_t NumBytes,
                                     OnReservedFunction OnReserved) {
   std::error_code EC;
InProcessMemoryMapper::~InProcessMemoryMapper() {
   }
 
   std::promise<MSVCPError> P;
   auto F = P.get_future();
   release(ReservationAddrs, [&](Error Err) { P.set_value(std::move(Err)); });
   cantFail(F.get());
 }
 
+// SharedMemoryMapper
+
+void SharedMemoryMapper::reserve(size_t NumBytes,
+                                 OnReservedFunction OnReserved) {
+#if defined(LLVM_ON_UNIX) || defined(_WIN32)
+
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceReserveSignature>(
+      SAs.Reserve,
+      [this, NumBytes, OnReserved = std::move(OnReserved)](
+          Error SerializationErr,
+          Expected<std::pair<ExecutorAddr, std::string>> Result) mutable {
+        if (SerializationErr) {
+          cantFail(Result.takeError());
+          return OnReserved(std::move(SerializationErr));
+        }
+
+        if (!Result)
+          return OnReserved(Result.takeError());
+
+        ExecutorAddr RemoteAddr;
+        std::string SharedMemoryName;
+        std::tie(RemoteAddr, SharedMemoryName) = std::move(*Result);
+
+        void *LocalAddr = nullptr;
+
+#if defined(LLVM_ON_UNIX)
+
+        int SharedMemoryFile = shm_open(SharedMemoryName.c_str(), O_RDWR, 0700);
+        if (SharedMemoryFile < 0) {
+          return OnReserved(errorCodeToError(
+              std::error_code(errno, std::generic_category())));
+        }
+
+        // this prevents other processes from accessing it by name
+        shm_unlink(SharedMemoryName.c_str());
+
+        LocalAddr = mmap(nullptr, NumBytes, PROT_READ | PROT_WRITE, MAP_SHARED,
+                         SharedMemoryFile, 0);
+        if (LocalAddr == MAP_FAILED) {
+          return OnReserved(errorCodeToError(
+              std::error_code(errno, std::generic_category())));
+        }
+
+        close(SharedMemoryFile);
+
+#elif defined(_WIN32)
+
+        std::wstring WideSharedMemoryName(SharedMemoryName.begin(),
+                                          SharedMemoryName.end());
+        HANDLE SharedMemoryFile = OpenFileMappingW(FILE_MAP_ALL_ACCESS, FALSE,
+                                                  WideSharedMemoryName.c_str());
+        if (!SharedMemoryFile)
+          return OnReserved(errorCodeToError(mapWindowsError(GetLastError())));
+
+        LocalAddr =
+            MapViewOfFile(SharedMemoryFile, FILE_MAP_ALL_ACCESS, 0, 0, 0);
+        if (!LocalAddr) {
+          CloseHandle(SharedMemoryFile);
+          return OnReserved(errorCodeToError(mapWindowsError(GetLastError())));
+        }
+
+        CloseHandle(SharedMemoryFile);
+
+#endif
+        {
+          std::lock_guard<std::mutex> Lock(Mutex);
+          Reservations.insert({RemoteAddr, {LocalAddr, NumBytes}});
+        }
+
+        OnReserved(ExecutorAddrRange(RemoteAddr, NumBytes));
+      },
+      SAs.Instance, static_cast<uint64_t>(NumBytes));
+
+#else
+  OnReserved(make_error<StringError>(
+      "SharedMemoryMapper is not supported on this platform yet",
+      inconvertibleErrorCode()));
+#endif
+}
+
+char *SharedMemoryMapper::prepare(ExecutorAddr Addr, size_t ContentSize) {
+  auto R = Reservations.upper_bound(Addr);
+  assert(R != Reservations.begin() && "Attempt to prepare unknown range");
+  R--;
+
+  ExecutorAddrDiff Offset = Addr - R->first;
+
+  return static_cast<char *>(R->second.LocalAddr) + Offset;
+}
+
+void SharedMemoryMapper::initialize(MemoryMapper::AllocInfo &AI,
+                                    OnInitializedFunction OnInitialized) {
+  auto Reservation = Reservations.find(AI.MappingBase);
+  assert(Reservation != Reservations.end() &&
+         "Attempt to initialize unreserved range");
+
+  tpctypes::SharedMemoryFinalizeRequest FR;
+
+  AI.Actions.swap(FR.Actions);
+
+  FR.Segments.reserve(AI.Segments.size());
+
+  for (auto Segment : AI.Segments) {
+    char *Base =
+        static_cast<char *>(Reservation->second.LocalAddr) + Segment.Offset;
+    std::memset(Base + Segment.ContentSize, 0, Segment.ZeroFillSize);
+
+    tpctypes::SharedMemorySegFinalizeRequest SegReq;
+    SegReq.Prot = tpctypes::toWireProtectionFlags(
+        static_cast<sys::Memory::ProtectionFlags>(Segment.Prot));
+    SegReq.Addr = AI.MappingBase + Segment.Offset;
+    SegReq.Size = Segment.ContentSize + Segment.ZeroFillSize;
+
+    FR.Segments.push_back(SegReq);
+  }
+
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceInitializeSignature>(
+      SAs.Initialize,
+      [OnInitialized = std::move(OnInitialized)](
+          Error SerializationErr, Expected<ExecutorAddr> Result) mutable {
+        if (SerializationErr) {
+          cantFail(Result.takeError());
+          return OnInitialized(std::move(SerializationErr));
+        }
+
+        OnInitialized(std::move(Result));
+      },
+      SAs.Instance, AI.MappingBase, std::move(FR));
+}
+
+void SharedMemoryMapper::deinitialize(
+    ArrayRef<ExecutorAddr> Allocations,
+    MemoryMapper::OnDeinitializedFunction OnDeinitialized) {
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceDeinitializeSignature>(
+      SAs.Deinitialize,
+      [OnDeinitialized = std::move(OnDeinitialized)](Error SerializationErr,
+                                                     Error Result) mutable {
+        if (SerializationErr) {
+          cantFail(std::move(Result));
+          return OnDeinitialized(std::move(SerializationErr));
+        }
+
+        OnDeinitialized(std::move(Result));
+      },
+      SAs.Instance, Allocations);
+}
+
+void SharedMemoryMapper::release(ArrayRef<ExecutorAddr> Bases,
+                                 OnReleasedFunction OnReleased) {
+#if defined(LLVM_ON_UNIX) || defined(_WIN32)
+  Error Err = Error::success();
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+
+    for (auto Base : Bases) {
+
+#if defined(LLVM_ON_UNIX)
+
+      if (munmap(Reservations[Base].LocalAddr, Reservations[Base].Size) != 0)
+        Err = joinErrors(std::move(Err), errorCodeToError(std::error_code(
+                                             errno, std::generic_category())));
+
+#elif defined(_WIN32)
+
+      if (!UnmapViewOfFile(Reservations[Base].LocalAddr))
+        joinErrors(std::move(Err),
+                   errorCodeToError(mapWindowsError(GetLastError())));
+
+#endif
+
+      Reservations.erase(Base);
+    }
+  }
+
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceReleaseSignature>(
+      SAs.Release,
+      [OnReleased = std::move(OnReleased),
+       Err = std::move(Err)](Error SerializationErr, Error Result) mutable {
+        if (SerializationErr) {
+          cantFail(std::move(Result));
+          return OnReleased(
+              joinErrors(std::move(Err), std::move(SerializationErr)));
+        }
+
+        return OnReleased(joinErrors(std::move(Err), std::move(Result)));
+      },
+      SAs.Instance, Bases);
+#else
+  OnReleased(make_error<StringError>(
+      "SharedMemoryMapper is not supported on this platform yet",
+      inconvertibleErrorCode()));
+#endif
+}
+
+SharedMemoryMapper::~SharedMemoryMapper() {
+  std::vector<ExecutorAddr> ReservationAddrs;
+  if (!Reservations.empty()) {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    {
+      ReservationAddrs.reserve(Reservations.size());
+      for (const auto &R : Reservations) {
+        ReservationAddrs.push_back(R.first);
+      }
+    }
+  }
+
+  std::promise<MSVCPError> P;
+  auto F = P.get_future();
+  release(ReservationAddrs, [&](Error Err) { P.set_value(std::move(Err)); });
+  // FIXME: Release can actually fail. The error should be propagated.
+  // Meanwhile, a better option is to explicitly call release().
+  cantFail(F.get());
+}
+
 } // namespace orc
 
 } // namespace llvm

llvm/lib/ExecutionEngine/Orc/Shared/OrcRTBridge.cpp

 namespace rt {
 
 const char *SimpleExecutorDylibManagerInstanceName =
     "__llvm_orc_SimpleExecutorDylibManager_Instance";
 const char *SimpleExecutorDylibManagerOpenWrapperName =
     "__llvm_orc_SimpleExecutorDylibManager_open_wrapper";
 const char *SimpleExecutorDylibManagerLookupWrapperName =
     "__llvm_orc_SimpleExecutorDylibManager_lookup_wrapper";
+
 const char *SimpleExecutorMemoryManagerInstanceName =
     "__llvm_orc_SimpleExecutorMemoryManager_Instance";
 const char *SimpleExecutorMemoryManagerReserveWrapperName =
     "__llvm_orc_SimpleExecutorMemoryManager_reserve_wrapper";
 const char *SimpleExecutorMemoryManagerFinalizeWrapperName =
     "__llvm_orc_SimpleExecutorMemoryManager_finalize_wrapper";
 const char *SimpleExecutorMemoryManagerDeallocateWrapperName =
     "__llvm_orc_SimpleExecutorMemoryManager_deallocate_wrapper";
+
+const char *ExecutorSharedMemoryMapperServiceInstanceName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Instance";
+const char *ExecutorSharedMemoryMapperServiceReserveWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Reserve";
+const char *ExecutorSharedMemoryMapperServiceInitializeWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Initialize";
+const char *ExecutorSharedMemoryMapperServiceDeinitializeWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Deinitialize";
+const char *ExecutorSharedMemoryMapperServiceReleaseWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Release";
+
 const char *MemoryWriteUInt8sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint8s_wrapper";
 const char *MemoryWriteUInt16sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint16s_wrapper";
 const char *MemoryWriteUInt32sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint32s_wrapper";
 const char *MemoryWriteUInt64sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint64s_wrapper";
 const char *MemoryWriteBuffersWrapperName =
     "__llvm_orc_bootstrap_mem_write_buffers_wrapper";
+
 const char *RegisterEHFrameSectionWrapperName =
     "__llvm_orc_bootstrap_register_ehframe_section_wrapper";
 const char *DeregisterEHFrameSectionWrapperName =
     "__llvm_orc_bootstrap_deregister_ehframe_section_wrapper";
+
 const char *RunAsMainWrapperName = "__llvm_orc_bootstrap_run_as_main_wrapper";
 
 } // end namespace rt
 } // end namespace orc
 } // end namespace llvm

llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt

+if( CMAKE_HOST_UNIX AND HAVE_LIBRT )
+  set(rt_lib rt)
+endif()
+
 add_llvm_component_library(LLVMOrcTargetProcess
+  ExecutorSharedMemoryMapperService.cpp
   JITLoaderGDB.cpp
   OrcRTBootstrap.cpp
   RegisterEHFrames.cpp
   SimpleExecutorDylibManager.cpp
   SimpleExecutorMemoryManager.cpp
   SimpleRemoteEPCServer.cpp
   TargetExecutionUtils.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/Orc
 
   LINK_LIBS
   ${LLVM_PTHREAD_LIB}
+  ${rt_lib}
 
   LINK_COMPONENTS
   OrcShared
   Support
   )

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

+//===---------- ExecutorSharedMemoryMapperService.cpp -----------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.h"
+
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+#include "llvm/Support/Process.h"
+#include "llvm/Support/WindowsError.h"
+
+#include <sstream>
+
+#if defined(LLVM_ON_UNIX)
+#include <errno.h>
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#endif
+
+#if defined(_WIN32)
+static DWORD getWindowsProtectionFlags(unsigned Flags) {
+  switch (Flags & llvm::sys::Memory::MF_RWE_MASK) {
+  case llvm::sys::Memory::MF_READ:
+    return PAGE_READONLY;
+  case llvm::sys::Memory::MF_WRITE:
+    // Note: PAGE_WRITE is not supported by VirtualProtect
+    return PAGE_READWRITE;
+  case llvm::sys::Memory::MF_READ | llvm::sys::Memory::MF_WRITE:
+    return PAGE_READWRITE;
+  case llvm::sys::Memory::MF_READ | llvm::sys::Memory::MF_EXEC:
+    return PAGE_EXECUTE_READ;
+  case llvm::sys::Memory::MF_READ | llvm::sys::Memory::MF_WRITE |
+      llvm::sys::Memory::MF_EXEC:
+    return PAGE_EXECUTE_READWRITE;
+  case llvm::sys::Memory::MF_EXEC:
+    return PAGE_EXECUTE;
+  default:
+    llvm_unreachable("Illegal memory protection flag specified!");
+  }
+  // Provide a default return value as required by some compilers.
+  return PAGE_NOACCESS;
+}
+#endif
+
+namespace llvm {
+namespace orc {
+namespace rt_bootstrap {
+
+Expected<std::pair<ExecutorAddr, std::string>>
+ExecutorSharedMemoryMapperService::reserve(uint64_t Size) {
+#if defined(LLVM_ON_UNIX) || defined(_WIN32)
+
+#if defined(LLVM_ON_UNIX)
+
+  std::string SharedMemoryName;
+  {
+    std::stringstream SharedMemoryNameStream;
+    SharedMemoryNameStream << "/jitlink_" << sys::Process::getProcessId() << '_'
+                           << (++SharedMemoryCount);
+    SharedMemoryName = SharedMemoryNameStream.str();
+  }
+
+  int SharedMemoryFile =
+      shm_open(SharedMemoryName.c_str(), O_RDWR | O_CREAT | O_EXCL, 0700);
+  if (SharedMemoryFile < 0)
+    return errorCodeToError(std::error_code(errno, std::generic_category()));
+
+  // by default size is 0
+  if (ftruncate(SharedMemoryFile, Size) < 0)
+    return errorCodeToError(std::error_code(errno, std::generic_category()));
+
+  void *Addr = mmap(nullptr, Size, PROT_NONE, MAP_SHARED, SharedMemoryFile, 0);
+  if (Addr == MAP_FAILED)
+    return errorCodeToError(std::error_code(errno, std::generic_category()));
+
+  close(SharedMemoryFile);
+
+#elif defined(_WIN32)
+
+  std::string SharedMemoryName;
+  {
+    std::stringstream SharedMemoryNameStream;
+    SharedMemoryNameStream << "jitlink_" << sys::Process::getProcessId() << '_'
+                           << (++SharedMemoryCount);
+    SharedMemoryName = SharedMemoryNameStream.str();
+  }
+
+  std::wstring WideSharedMemoryName(SharedMemoryName.begin(),
+                                    SharedMemoryName.end());
+  HANDLE SharedMemoryFile = CreateFileMappingW(
+      INVALID_HANDLE_VALUE, NULL, PAGE_EXECUTE_READWRITE, Size >> 32,
+      Size & 0xffffffff, WideSharedMemoryName.c_str());
+  if (!SharedMemoryFile)
+    return errorCodeToError(mapWindowsError(GetLastError()));
+
+  void *Addr = MapViewOfFile(SharedMemoryFile,
+                             FILE_MAP_ALL_ACCESS | FILE_MAP_EXECUTE, 0, 0, 0);
+  if (!Addr) {
+    CloseHandle(SharedMemoryFile);
+    return errorCodeToError(mapWindowsError(GetLastError()));
+  }
+
+#endif
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    Reservations[Addr].Size = Size;
+#if defined(_WIN32)
+    Reservations[Addr].SharedMemoryFile = SharedMemoryFile;
+#endif
+  }
+
+  return std::make_pair(ExecutorAddr::fromPtr(Addr),
+                        std::move(SharedMemoryName));
+#else
+  return make_error<StringError>(
+      "SharedMemoryMapper is not supported on this platform yet",
+      inconvertibleErrorCode());
+#endif
+}
+
+Expected<ExecutorAddr> ExecutorSharedMemoryMapperService::initialize(
+    ExecutorAddr Reservation, tpctypes::SharedMemoryFinalizeRequest &FR) {
+#if defined(LLVM_ON_UNIX) || defined(_WIN32)
+
+  ExecutorAddr MinAddr(~0ULL);
+
+  // Contents are already in place
+  for (auto &Segment : FR.Segments) {
+    if (Segment.Addr < MinAddr)
+      MinAddr = Segment.Addr;
+
+#if defined(LLVM_ON_UNIX)
+
+    int NativeProt = 0;
+    if (Segment.Prot & tpctypes::WPF_Read)
+      NativeProt |= PROT_READ;
+    if (Segment.Prot & tpctypes::WPF_Write)
+      NativeProt |= PROT_WRITE;
+    if (Segment.Prot & tpctypes::WPF_Exec)
+      NativeProt |= PROT_EXEC;
+
+    if (mprotect(Segment.Addr.toPtr<void *>(), Segment.Size, NativeProt))
+      return errorCodeToError(std::error_code(errno, std::generic_category()));
+
+#elif defined(_WIN32)
+
+    DWORD NativeProt =
+        getWindowsProtectionFlags(fromWireProtectionFlags(Segment.Prot));
+
+    if (!VirtualProtect(Segment.Addr.toPtr<void *>(), Segment.Size, NativeProt,
+                        &NativeProt))
+      return errorCodeToError(mapWindowsError(GetLastError()));
+
+#endif
+
+    if (Segment.Prot & tpctypes::WPF_Exec)
+      sys::Memory::InvalidateInstructionCache(Segment.Addr.toPtr<void *>(),
+                                              Segment.Size);
+  }
+
+  // Run finalization actions and get deinitlization action list.
+  auto DeinitializeActions = shared::runFinalizeActions(FR.Actions);
+  if (!DeinitializeActions) {
+    return DeinitializeActions.takeError();
+  }
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    Allocations[MinAddr].DeinitializationActions =
+        std::move(*DeinitializeActions);
+    Reservations[Reservation.toPtr<void *>()].Allocations.push_back(MinAddr);
+  }
+
+  return MinAddr;
+
+#else
+  return make_error<StringError>(
+      "SharedMemoryMapper is not supported on this platform yet",
+      inconvertibleErrorCode());
+#endif
+}
+
+Error ExecutorSharedMemoryMapperService::deinitialize(
+    const std::vector<ExecutorAddr> &Bases) {
+  Error AllErr = Error::success();
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+
+    for (auto Base : Bases) {
+      if (Error Err = shared::runDeallocActions(
+              Allocations[Base].DeinitializationActions)) {
+        AllErr = joinErrors(std::move(AllErr), std::move(Err));
+      }
+
+      Allocations.erase(Base);
+    }
+  }
+
+  return AllErr;
+}
+
+Error ExecutorSharedMemoryMapperService::release(
+    const std::vector<ExecutorAddr> &Bases) {
+#if defined(LLVM_ON_UNIX) || defined(_WIN32)
+  Error Err = Error::success();
+
+  for (auto Base : Bases) {
+    std::vector<ExecutorAddr> AllocAddrs;
+    size_t Size;
+
+#if defined(_WIN32)
+    HANDLE SharedMemoryFile;
+#endif
+
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      auto &R = Reservations[Base.toPtr<void *>()];
+      Size = R.Size;
+
+#if defined(_WIN32)
+      SharedMemoryFile = R.SharedMemoryFile;
+#endif
+
+      AllocAddrs.swap(R.Allocations);
+    }
+
+    // deinitialize sub allocations
+    if (Error E = deinitialize(AllocAddrs))
+      Err = joinErrors(std::move(Err), std::move(E));
+
+#if defined(LLVM_ON_UNIX)
+
+    if (munmap(Base.toPtr<void *>(), Size) != 0)
+      Err = joinErrors(std::move(Err), errorCodeToError(std::error_code(
+                                           errno, std::generic_category())));
+
+#elif defined(_WIN32)
+
+    if (!UnmapViewOfFile(Base.toPtr<void *>()))
+      Err = joinErrors(std::move(Err),
+                       errorCodeToError(mapWindowsError(GetLastError())));
+
+    CloseHandle(SharedMemoryFile);
+
+#endif
+
+    std::lock_guard<std::mutex> Lock(Mutex);
+    Reservations.erase(Base.toPtr<void *>());
+  }
+
+  return Err;
+#else
+  return make_error<StringError>(
+      "SharedMemoryMapper is not supported on this platform yet",
+      inconvertibleErrorCode());
+#endif
+}
+
+Error ExecutorSharedMemoryMapperService::shutdown() {
+  std::vector<ExecutorAddr> ReservationAddrs;
+  if (!Reservations.empty()) {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    {
+      ReservationAddrs.reserve(Reservations.size());
+      for (const auto &R : Reservations) {
+        ReservationAddrs.push_back(ExecutorAddr::fromPtr(R.getFirst()));
+      }
+    }
+  }
+  return release(ReservationAddrs);
+
+  return Error::success();
+}
+
+void ExecutorSharedMemoryMapperService::addBootstrapSymbols(
+    StringMap<ExecutorAddr> &M) {
+  M[rt::ExecutorSharedMemoryMapperServiceInstanceName] =
+      ExecutorAddr::fromPtr(this);
+  M[rt::ExecutorSharedMemoryMapperServiceReserveWrapperName] =
+      ExecutorAddr::fromPtr(&reserveWrapper);
+  M[rt::ExecutorSharedMemoryMapperServiceInitializeWrapperName] =
+      ExecutorAddr::fromPtr(&initializeWrapper);
+  M[rt::ExecutorSharedMemoryMapperServiceDeinitializeWrapperName] =
+      ExecutorAddr::fromPtr(&deinitializeWrapper);
+  M[rt::ExecutorSharedMemoryMapperServiceReleaseWrapperName] =
+      ExecutorAddr::fromPtr(&releaseWrapper);
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::reserveWrapper(const char *ArgData,
+                                                  size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceReserveSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::reserve))
+          .release();
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::initializeWrapper(const char *ArgData,
+                                                     size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceInitializeSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::initialize))
+          .release();
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::deinitializeWrapper(const char *ArgData,
+                                                       size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceDeinitializeSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::deinitialize))
+          .release();
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::releaseWrapper(const char *ArgData,
+                                                  size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceReleaseSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::release))
+          .release();
+}
+
+} // namespace rt_bootstrap
+} // end namespace orc
+} // end namespace llvm

llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt

add_llvm_unittest(OrcJITTests
   LazyCallThroughAndReexportsTest.cpp
   LookupAndRecordAddrsTest.cpp
   MemoryMapperTest.cpp
   ObjectLinkingLayerTest.cpp
   OrcCAPITest.cpp
   OrcTestCommon.cpp
   ResourceTrackerTest.cpp
   RTDyldObjectLinkingLayerTest.cpp
+  SharedMemoryMapperTest.cpp
   SimpleExecutorMemoryManagerTest.cpp
   SimplePackedSerializationTest.cpp
   SymbolStringPoolTest.cpp
   TaskDispatchTest.cpp
   ThreadSafeModuleTest.cpp
   WrapperFunctionUtilsTest.cpp
   )
 
 target_link_libraries(OrcJITTests PRIVATE
                         LLVMTestingSupport
                         ${ORC_JIT_TEST_LIBS})
 
 set_property(TARGET OrcJITTests PROPERTY FOLDER "Tests/UnitTests/ExecutionTests")

llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp

+//===- SharedMemoryMapperTest.cpp -- Tests for SharedMemoryMapper ---------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "OrcTestCommon.h"
+
+#include "llvm/ExecutionEngine/Orc/MemoryMapper.h"
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+#include "llvm/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.h"
+#include "llvm/Testing/Support/Error.h"
+
+using namespace llvm;
+using namespace llvm::orc;
+using namespace llvm::orc::shared;
+using namespace llvm::orc::rt_bootstrap;
+
+#if defined(LLVM_ON_UNIX) || defined(_WIN32)
+
+// A basic function to be used as both initializer/deinitializer
+orc::shared::CWrapperFunctionResult incrementWrapper(const char *ArgData,
+                                                     size_t ArgSize) {
+  return WrapperFunction<SPSError(SPSExecutorAddr)>::handle(
+             ArgData, ArgSize,
+             [](ExecutorAddr A) -> Error {
+               *A.toPtr<int *>() += 1;
+               return Error::success();
+             })
+      .release();
+}
+
+TEST(SharedMemoryMapperTest, MemReserveInitializeDeinitializeRelease) {
+  // These counters are used to track how many times the initializer and
+  // deinitializer functions are called
+  int InitializeCounter = 0;
+  int DeinitializeCounter = 0;
+
+  auto SelfEPC = cantFail(SelfExecutorProcessControl::Create());
+
+  ExecutorSharedMemoryMapperService MapperService;
+
+  SharedMemoryMapper::SymbolAddrs SAs;
+  {
+    StringMap<ExecutorAddr> Map;
+    MapperService.addBootstrapSymbols(Map);
+    SAs.Instance = Map[rt::ExecutorSharedMemoryMapperServiceInstanceName];
+    SAs.Reserve = Map[rt::ExecutorSharedMemoryMapperServiceReserveWrapperName];
+    SAs.Initialize =
+        Map[rt::ExecutorSharedMemoryMapperServiceInitializeWrapperName];
+    SAs.Deinitialize =
+        Map[rt::ExecutorSharedMemoryMapperServiceDeinitializeWrapperName];
+    SAs.Release = Map[rt::ExecutorSharedMemoryMapperServiceReleaseWrapperName];
+  }
+
+  std::string TestString = "Hello, World!";
+
+  // barrier
+  std::promise<void> P;
+  auto F = P.get_future();
+
+  {
+    auto PageSize = cantFail(sys::Process::getPageSize());
+    size_t ReqSize = PageSize;
+
+    std::unique_ptr<MemoryMapper> Mapper =
+        std::make_unique<SharedMemoryMapper>(*SelfEPC, SAs);
+
+    Mapper->reserve(ReqSize, [&](Expected<ExecutorAddrRange> Result) {
+      EXPECT_THAT_ERROR(Result.takeError(), Succeeded());
+      auto Reservation = std::move(*Result);
+      {
+        char *Addr = Mapper->prepare(Reservation.Start, TestString.size() + 1);
+        std::strcpy(Addr, TestString.c_str());
+      }
+      MemoryMapper::AllocInfo AI;
+      {
+        MemoryMapper::AllocInfo::SegInfo SI;
+        SI.Offset = 0;
+        SI.ContentSize = TestString.size() + 1;
+        SI.ZeroFillSize = PageSize - SI.ContentSize;
+        SI.Prot = sys::Memory::MF_READ | sys::Memory::MF_WRITE;
+
+        AI.MappingBase = Reservation.Start;
+        AI.Segments.push_back(SI);
+        AI.Actions.push_back(
+            {cantFail(WrapperFunctionCall::Create<SPSArgList<SPSExecutorAddr>>(
+                 ExecutorAddr::fromPtr(incrementWrapper),
+                 ExecutorAddr::fromPtr(&InitializeCounter))),
+             cantFail(WrapperFunctionCall::Create<SPSArgList<SPSExecutorAddr>>(
+                 ExecutorAddr::fromPtr(incrementWrapper),
+                 ExecutorAddr::fromPtr(&DeinitializeCounter)))});
+      }
+
+      EXPECT_EQ(InitializeCounter, 0);
+      EXPECT_EQ(DeinitializeCounter, 0);
+
+      Mapper->initialize(AI, [&, Reservation](Expected<ExecutorAddr> Result) {
+        EXPECT_THAT_ERROR(Result.takeError(), Succeeded());
+
+        EXPECT_EQ(TestString, std::string(static_cast<char *>(
+                                  Reservation.Start.toPtr<char *>())));
+
+        EXPECT_EQ(InitializeCounter, 1);
+        EXPECT_EQ(DeinitializeCounter, 0);
+
+        Mapper->deinitialize({*Result}, [&, Reservation](Error Err) {
+          EXPECT_THAT_ERROR(std::move(Err), Succeeded());
+
+          EXPECT_EQ(InitializeCounter, 1);
+          EXPECT_EQ(DeinitializeCounter, 1);
+
+          Mapper->release({Reservation.Start}, [&](Error Err) {
+            EXPECT_THAT_ERROR(std::move(Err), Succeeded());
+
+            P.set_value();
+          });
+        });
+      });
+    });
+
+    // This will block the test if any of the above callbacks are not executed
+    F.wait();
+    // Mapper must be destructed before calling shutdown to avoid double free
+  }
+
+  EXPECT_THAT_ERROR(MapperService.shutdown(), Succeeded());
+  cantFail(SelfEPC->disconnect());
+}
+
+#endif