[Orc][JITLink] Add a shared memory based implementation of MemoryMapper

[argentite]

Summary

This is an implementation of orc::MemoryMapper that maps shared memory
pages in both executor and controller process and writes directly to
them avoiding transferring content over EPC.
All allocations are properly deinitialized automatically on the
executor side at shutdown by the ExecutorSharedMemoryMapperService.

[argentite]

Windows support and some other fixes

[argentite]

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

26–49

This function is copied from lib/Support/Windows/Memory.inc. We probably should expose it from there instead.

[sgraenitz]

Thanks for posting your patch! I will continue reviewing, but here is a first detail I found..

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

225

I had to cast this to uint64_t explicitly in order to fix a SPS encoding error. It appears the issue is that on my system size_t is unsigned long and not unsigned long long.

[argentite]

Added explicit cast from size_t to uint64_t

[lhames]

This looks great. Thanks @argentite!

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

358

The call to release could fail -- we'll want to figure out the right error plumbing for this in the long run. Could you add a // FIXME for now?

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

157

Are there any other guards to prevent ExecutorSharedMemoryMapperService from being instantiated / used on unsupported platforms? If not, I think this should return an llvm::Error for now.

llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp

59–64

Looks like these could be moved into the scope below.

60–61

There's no explicit test that the lambdas are executed, but I think the use of this promise/future pair in the innermost lambda is sufficient -- the test will block unless all lambdas are run. That's probably worth a comment to point out though.

[argentite]

Replaced some llvm_unreachable() with llvm::Error

[argentite]

Skip the test on unimplemented operating systems

[amyk]

Hi there!

I just wanted to give a heads up that it looks like this patch is causing failures on some of the PPC bots that build with shared libraries:
https://lab.llvm.org/buildbot/#/builders/57
https://lab.llvm.org/buildbot/#/builders/121

The error looks like this in https://lab.llvm.org/buildbot/#/builders/121/builds/21343/steps/4/logs/stdio:

[124/1619] Linking CXX shared library lib/libLLVMOrcTargetProcess.so.15git
FAILED: lib/libLLVMOrcTargetProcess.so.15git 
: && /usr/lib64/ccache/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/buildbots/ppc64le-clang-multistage-test/clang-ppc64le-multistage/stage1/./lib  -Wl,--gc-sections -shared -Wl,-soname,libLLVMOrcTargetProcess.so.15git -o lib/libLLVMOrcTargetProcess.so.15git lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/ExecutorSharedMemoryMapperService.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/JITLoaderGDB.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/OrcRTBootstrap.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/RegisterEHFrames.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/SimpleExecutorDylibManager.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/SimpleExecutorMemoryManager.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/SimpleRemoteEPCServer.cpp.o lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/TargetExecutionUtils.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  -lpthread  lib/libLLVMOrcShared.so.15git  lib/libLLVMSupport.so.15git  -Wl,-rpath-link,/home/buildbots/ppc64le-clang-multistage-test/clang-ppc64le-multistage/stage1/lib && :
lib/ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/ExecutorSharedMemoryMapperService.cpp.o: In function `llvm::orc::rt_bootstrap::ExecutorSharedMemoryMapperService::reserve[abi:cxx11](unsigned long)':
ExecutorSharedMemoryMapperService.cpp:(.text._ZN4llvm3orc12rt_bootstrap33ExecutorSharedMemoryMapperService7reserveB5cxx11Em+0x398): undefined reference to `shm_open'
collect2: error: ld returned 1 exit status

Are you able to look into this?

Thanks.

[argentite]

Hi! I am not very familiar with powerpc but it seems like POSIX shared memory APIs should be available.
From this page I suspect shm_ family of functions could be in librt instead of libc.

[abinavpp]

In D128544#3650815, @argentite wrote:

Hi! I am not very familiar with powerpc but it seems like POSIX shared memory APIs should be available.
From this page I suspect shm_ family of functions could be in librt instead of libc.

I don't think this is specific to PPC, shm_* APIs are in librt right? I can see "Link with -lrt." in the synopsis of https://man7.org/linux/man-pages/man3/shm_open.3.html. So I guess doing something like:

diff --git a/llvm/lib/ExecutionEngine/CMakeLists.txt b/llvm/lib/ExecutionEngine/CMakeLists.txt
index a5607872c706..cdb7b3e622c3 100644
--- a/llvm/lib/ExecutionEngine/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/CMakeLists.txt
@@ -24,7 +24,7 @@ add_llvm_component_library(LLVMExecutionEngine
   )
 
 if(BUILD_SHARED_LIBS)
-  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld)
+  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld rt)
 endif()
 
 add_subdirectory(Interpreter)
diff --git a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
index 415876abd65f..9a2709a4d6af 100644
--- a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
@@ -12,7 +12,7 @@ add_llvm_component_library(LLVMOrcTargetProcess
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/Orc
 
   LINK_LIBS
-  ${LLVM_PTHREAD_LIB}
+  ${LLVM_PTHREAD_LIB} rt
 
   LINK_COMPONENTS
   OrcShared

should get the shared-lib build fixed in Linux, but I'm not sure about other platforms.

[c-rhodes]

I've reverted this in 3e9cc543f223 since I'm hitting the same error as @amyk compiling with shared libs on AArch64 Linux. I had to revert a fix (46b1a7c5f9e6) from @mstorsjo for mingw as well so you probably want to pick that up when re-landing.

[thakis]

In D128544#3650873, @abinavpp wrote:

In D128544#3650815, @argentite wrote:

Hi! I am not very familiar with powerpc but it seems like POSIX shared memory APIs should be available.
From this page I suspect shm_ family of functions could be in librt instead of libc.

I don't think this is specific to PPC, shm_* APIs are in librt right? I can see "Link with -lrt." in the synopsis of https://man7.org/linux/man-pages/man3/shm_open.3.html. So I guess doing something like:

diff --git a/llvm/lib/ExecutionEngine/CMakeLists.txt b/llvm/lib/ExecutionEngine/CMakeLists.txt
index a5607872c706..cdb7b3e622c3 100644
--- a/llvm/lib/ExecutionEngine/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/CMakeLists.txt
@@ -24,7 +24,7 @@ add_llvm_component_library(LLVMExecutionEngine
   )
 
 if(BUILD_SHARED_LIBS)
-  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld)
+  target_link_libraries(LLVMExecutionEngine PUBLIC LLVMRuntimeDyld rt)
 endif()
 
 add_subdirectory(Interpreter)
diff --git a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
index 415876abd65f..9a2709a4d6af 100644
--- a/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
+++ b/llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
@@ -12,7 +12,7 @@ add_llvm_component_library(LLVMOrcTargetProcess
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/ExecutionEngine/Orc
 
   LINK_LIBS
-  ${LLVM_PTHREAD_LIB}
+  ${LLVM_PTHREAD_LIB} rt
 
   LINK_COMPONENTS
   OrcShared

should get the shared-lib build fixed in Linux, but I'm not sure about other platforms.

This looks right to me, but it shouldn't be done on windows and apple platforms i think. (Not sure about the bsds.)

Maybe look for what we do for HAVE_LIBRT in llvm/cmake/config-ix.cmake and set a var like LLVM_RT_LIB there and use that here (similar to LLVM_PTHREAD_LIB).

[argentite]

Link with librt if available

I think this should work everywhere.

It seems like some platforms have shm_ functions in libc and librt acts like a placeholder.

[c-rhodes]

In D128544#3651546, @argentite wrote:

Link with librt if available

I think this should work everywhere.

It seems like some platforms have shm_ functions in libc and librt acts like a placeholder.

This compiles for me now, cheers

[lhames]

LGTM otherwise -- thanks for chasing up the fix @argentite.

llvm/lib/ExecutionEngine/Orc/CMakeLists.txt

2 ↗

(On Diff #444608)

As @thakis suggested I think this should be LLVM_RT_LIB (for consistency with LLVM_PTHREAD_LIB).

[lhames]

As @thakis suggested I think this should be LLVM_RT_LIB (for consistency with LLVM_PTHREAD_LIB).

Re-committed on @argentite's behalf. I ended up leaving this as-is: it looks like libSupport currently does something similar with a local CMake variable for rt. We can introduce LLVM_RT_LIB in a follow-up patch if we decide it's worth it.

[hctim]

Looks like this broke the UBSan buildbot:

https://lab.llvm.org/buildbot/#/builders/85/builds/9186

Details on how to reproduce sanitizer buildbots can be found at https://github.com/google/sanitizers/wiki/SanitizerBotReproduceBuild, but hopefully the error from the bot should be enough to identify the issue.

-- Testing: 67188 tests, 80 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90
FAIL: LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/84/128 (61930 of 67188)
******************** TEST 'LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/84/128' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm_build_ubsan/unittests/ExecutionEngine/Orc/./OrcJITTests-LLVM-Unit-4179923-84-128.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=128 GTEST_SHARD_INDEX=84 /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm_build_ubsan/unittests/ExecutionEngine/Orc/./OrcJITTests
--
Note: This is test shard 85 of 128.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from SharedMemoryMapperTest
[ RUN      ] SharedMemoryMapperTest.MemReserveInitializeDeinitializeRelease
/b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:61:20: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/lib/gcc-cross/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include/string.h:44:28: note: nonnull attribute specified here
    #0 0x5567ac3b1159 in llvm::orc::shared::SPSOutputBuffer::write(char const*, unsigned long) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:61:5
    #1 0x5567ac74da31 in serialize /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:352:15
    #2 0x5567ac74da31 in serialize<llvm::ArrayRef<char> > /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #3 0x5567ac74da31 in bool llvm::orc::shared::SPSArgList<unsigned long, llvm::orc::shared::SPSSequence<char>>::serialize<unsigned long, llvm::ArrayRef<char>>(llvm::orc::shared::SPSOutputBuffer&, unsigned long const&, llvm::ArrayRef<char> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #4 0x5567ac74d9a4 in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>::serialize<llvm::orc::ExecutorAddr, unsigned long, llvm::ArrayRef<char>>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::ExecutorAddr const&, unsigned long const&, llvm::ArrayRef<char> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #5 0x5567ac74d95d in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>::serialize<llvm::orc::tpctypes::WireProtectionFlags, llvm::orc::ExecutorAddr, unsigned long, llvm::ArrayRef<char>>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::tpctypes::WireProtectionFlags const&, llvm::orc::ExecutorAddr const&, unsigned long const&, llvm::ArrayRef<char> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #6 0x5567ac74d8a8 in serialize<llvm::orc::tpctypes::SegFinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #7 0x5567ac74d8a8 in llvm::orc::shared::SPSSerializationTraits<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, std::__1::vector<llvm::orc::tpctypes::SegFinalizeRequest, std::__1::allocator<llvm::orc::tpctypes::SegFinalizeRequest>>, void>::serialize(llvm::orc::shared::SPSOutputBuffer&, std::__1::vector<llvm::orc::tpctypes::SegFinalizeRequest, std::__1::allocator<llvm::orc::tpctypes::SegFinalizeRequest>> const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:385:12
    #8 0x5567ac74d65f in serialize<std::__1::vector<llvm::orc::tpctypes::SegFinalizeRequest, std::__1::allocator<llvm::orc::tpctypes::SegFinalizeRequest> >, std::__1::vector<llvm::orc::shared::AllocActionCallPair, std::__1::allocator<llvm::orc::shared::AllocActionCallPair> > > /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #9 0x5567ac74d65f in serialize /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h:199:12
    #10 0x5567ac74d65f in serialize<llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:141:12
    #11 0x5567ac74d65f in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>>>>>::serialize<llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::ExecutorAddr const&, llvm::orc::tpctypes::FinalizeRequest const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #12 0x5567ac76cbc4 in bool llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>>>>>::serialize<llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest>(llvm::orc::shared::SPSOutputBuffer&, llvm::orc::ExecutorAddr const&, llvm::orc::ExecutorAddr const&, llvm::orc::tpctypes::FinalizeRequest const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:142:12
    #13 0x5567ac76c8cb in llvm::orc::shared::WrapperFunctionResult llvm::orc::shared::detail::serializeViaSPSToWrapperFunctionResult<llvm::orc::shared::SPSArgList<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char>>>, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>>>>>, llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest>(llvm::orc::ExecutorAddr const&, llvm::orc::ExecutorAddr const&, llvm::orc::tpctypes::FinalizeRequest const&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h:176:8
    #14 0x5567ac7685c4 in callAsync<(lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:317:9), (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:281:7), llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h:484:9
    #15 0x5567ac7685c4 in callSPSWrapperAsync<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSExecutorAddr> (llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char> > >, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> >, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> > > > >), llvm::orc::ExecutorProcessControl::RunAsTask, (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:281:7), llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:316:5
    #16 0x5567ac7685c4 in callSPSWrapperAsync<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSExecutorAddr> (llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSMemoryProtectionFlags, llvm::orc::shared::SPSExecutorAddr, unsigned long, llvm::orc::shared::SPSSequence<char> > >, llvm::orc::shared::SPSSequence<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> >, llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char> > > > >), (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:281:7), llvm::orc::ExecutorAddr, llvm::orc::ExecutorAddr, llvm::orc::tpctypes::FinalizeRequest> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:331:5
    #17 0x5567ac7685c4 in llvm::orc::SharedMemoryMapper::initialize(llvm::orc::MemoryMapper::AllocInfo&, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddr>)>) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:278:7
    #18 0x5567ac3e1c28 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp:100:15
    #19 0x5567ac3e1c28 in void llvm::detail::UniqueFunctionBase<void, llvm::Expected<llvm::orc::ExecutorAddrRange>>::CallImpl<SharedMemoryMapperTest_MemReserveInitializeDeinitializeRelease_Test::TestBody()::$_1>(void*, llvm::Expected<llvm::orc::ExecutorAddrRange>&) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ADT/FunctionExtras.h:222:12
    #20 0x5567ac76b10a in llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2::operator()(llvm::Error, llvm::Expected<std::__1::pair<llvm::orc::ExecutorAddr, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp:231:9
    #21 0x5567ac76be35 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h
    #22 0x5567ac76be35 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ExecutorProcessControl.h:92:23
    #23 0x5567ac76be35 in llvm::orc::GenericNamedTaskImpl<llvm::orc::ExecutorProcessControl::IncomingWFRHandler llvm::orc::ExecutorProcessControl::RunAsTask::operator()<void llvm::orc::shared::WrapperFunction<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>> (llvm::orc::shared::SPSExecutorAddr, unsigned long)>::callAsync<void llvm::orc::ExecutorProcessControl::callSPSWrapperAsync<llvm::orc::shared::SPSExpected<llvm::orc::shared::SPSTuple<llvm::orc::shared::SPSExecutorAddr, llvm::orc::shared::SPSSequence<char>>> (llvm::orc::shared::SPSExecutorAddr, unsigned long), llvm::orc::ExecutorProcessControl::RunAsTask, llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2, llvm::orc::ExecutorAddr, unsigned long>(llvm::orc::ExecutorProcessControl::RunAsTask&&, llvm::orc::ExecutorAddr, llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2&&, llvm::orc::ExecutorAddr const&, unsigned long const&)::'lambda'(auto&&, char const*, unsigned long), llvm::orc::SharedMemoryMapper::reserve(unsigned long, llvm::unique_function<void (llvm::Expected<llvm::orc::ExecutorAddrRange>)>)::$_2, llvm::orc::ExecutorAddr, unsigned long>(auto&&, llvm::orc::ExecutorProcessControl::RunAsTask&&, llvm::orc::ExecutorAddr const&, unsigned long const&)::'lambda'(llvm::orc::shared::WrapperFunctionResult)>(auto&&)::'lambda'(llvm::orc::shared::WrapperFunctionResult)::operator()(llvm::orc::shared::WrapperFunctionResult)::'lambda'()>::run() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/TaskDispatch.h:68:25
    #24 0x5567ac79f190 in operator() /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/TaskDispatch.cpp:33:8
    #25 0x5567ac79f190 in __invoke<(lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/TaskDispatch.cpp:32:15)> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/libcxx_build_ubsan/include/c++/v1/__functional/invoke.h:403:23
    #26 0x5567ac79f190 in __thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, (lambda at /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/lib/ExecutionEngine/Orc/TaskDispatch.cpp:32:15)> /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/libcxx_build_ubsan/include/c++/v1/thread:284:5
    #27 0x5567ac79f190 in void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, llvm::orc::DynamicThreadPoolTaskDispatcher::dispatch(std::__1::unique_ptr<llvm::orc::Task, std::__1::default_delete<llvm::orc::Task>>)::$_0>>(void*) /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/libcxx_build_ubsan/include/c++/v1/thread:295:5
    #28 0x7fc9f385eb42  (/lib/x86_64-linux-gnu/libc.so.6+0x94b42) (BuildId: 89c3cb85f9e55046776471fed05ec441581d1969)
    #29 0x7fc9f38f09ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff) (BuildId: 89c3cb85f9e55046776471fed05ec441581d1969)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /b/sanitizer-x86_64-linux-bootstrap-ubsan/build/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h:61:20 in

[argentite]

Use new SharedMemorySegFinalizeRequest instead of SegFinalizeRequest

The problem seems to be that there is a Content: ArrayRef<char> in SegFinalizeRequest. In SharedMemoryMapper it is set to (nullptr, 0) since we do not use it but the serialization code runs memcpy anyway with source nullptr and length 0. A new type SharedMemorySegFinalizeRequest without this field will make the intent clear as well.

[lhames]

In D128544#3659757, @argentite wrote:

Use new SharedMemorySegFinalizeRequest instead of SegFinalizeRequest

The problem seems to be that there is a Content: ArrayRef<char> in SegFinalizeRequest. In SharedMemoryMapper it is set to (nullptr, 0) since we do not use it but the serialization code runs memcpy anyway with source nullptr and length 0. A new type SharedMemorySegFinalizeRequest without this field will make the intent clear as well.

I fixed serialization of empty ArrayRef<char> in 67220c2ad72, but I like the idea of a custom request type for the SharedMemoryMapper too -- I'll keep that and re-re-land this.

[lhames]

Re-re-committed in https://reviews.llvm.org/rG1b1f1c778695442c2683813ec7d4b557bb279e26.

[RKSimon]

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

279

@argentite Static analysis is complaining that this second return is dead code - is it or should the return release() be inside the if() ?

[argentite]

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

279

Yup that release should be inside the if body but we also can't hold the mutex while calling release. I reordered them here: https://reviews.llvm.org/D131510 .

llvm/include/llvm/ExecutionEngine/Orc/MemoryMapper.h

private:
   };
   using ReservationMap = DenseMap<void *, Reservation>;
 
   std::mutex Mutex;
   ReservationMap Reservations;
   AllocationMap Allocations;
 };
 
+class SharedMemoryMapper final : public MemoryMapper {
+public:
+  struct SymbolAddrs {
+    ExecutorAddr Instance;
+    ExecutorAddr Reserve;
+    ExecutorAddr Initialize;
+    ExecutorAddr Deinitialize;
+    ExecutorAddr Release;
+  };
+
+  SharedMemoryMapper(ExecutorProcessControl &EPC, SymbolAddrs SAs)
+      : EPC(EPC), SAs(SAs) {}
+
+  void reserve(size_t NumBytes, OnReservedFunction OnReserved) override;
+
+  char *prepare(ExecutorAddr Addr, size_t ContentSize) override;
+
+  void initialize(AllocInfo &AI, OnInitializedFunction OnInitialized) override;
+
+  void deinitialize(ArrayRef<ExecutorAddr> Allocations,
+                    OnDeinitializedFunction OnDeInitialized) override;
+
+  void release(ArrayRef<ExecutorAddr> Reservations,
+               OnReleasedFunction OnRelease) override;
+
+  ~SharedMemoryMapper() override;
+
+private:
+  struct Reservation {
+    void *LocalAddr;
+    size_t Size;
+  };
+
+  ExecutorProcessControl &EPC;
+  SymbolAddrs SAs;
+
+  std::mutex Mutex;
+
+  std::map<ExecutorAddr, Reservation> Reservations;
+};
+
 } // namespace orc
 } // end namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_ORC_MEMORYMAPPER_H

llvm/include/llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h

 extern const char *SimpleExecutorDylibManagerOpenWrapperName;
 extern const char *SimpleExecutorDylibManagerLookupWrapperName;
 
 extern const char *SimpleExecutorMemoryManagerInstanceName;
 extern const char *SimpleExecutorMemoryManagerReserveWrapperName;
 extern const char *SimpleExecutorMemoryManagerFinalizeWrapperName;
 extern const char *SimpleExecutorMemoryManagerDeallocateWrapperName;
 
+extern const char *ExecutorSharedMemoryMapperServiceInstanceName;
+extern const char *ExecutorSharedMemoryMapperServiceReserveWrapperName;
+extern const char *ExecutorSharedMemoryMapperServiceInitializeWrapperName;
+extern const char *ExecutorSharedMemoryMapperServiceDeinitializeWrapperName;
+extern const char *ExecutorSharedMemoryMapperServiceReleaseWrapperName;
+
 extern const char *MemoryWriteUInt8sWrapperName;
 extern const char *MemoryWriteUInt16sWrapperName;
 extern const char *MemoryWriteUInt32sWrapperName;
 extern const char *MemoryWriteUInt64sWrapperName;
 extern const char *MemoryWriteBuffersWrapperName;
 
 extern const char *RegisterEHFrameSectionWrapperName;
 extern const char *DeregisterEHFrameSectionWrapperName;

 using SPSSimpleExecutorMemoryManagerReserveSignature =
     shared::SPSExpected<shared::SPSExecutorAddr>(shared::SPSExecutorAddr,
                                                  uint64_t);
 using SPSSimpleExecutorMemoryManagerFinalizeSignature =
     shared::SPSError(shared::SPSExecutorAddr, shared::SPSFinalizeRequest);
 using SPSSimpleExecutorMemoryManagerDeallocateSignature = shared::SPSError(
     shared::SPSExecutorAddr, shared::SPSSequence<shared::SPSExecutorAddr>);
 
+// ExecutorSharedMemoryMapperService
+using SPSExecutorSharedMemoryMapperServiceReserveSignature =
+    shared::SPSExpected<
+        shared::SPSTuple<shared::SPSExecutorAddr, shared::SPSString>>(
+        shared::SPSExecutorAddr, uint64_t);
+using SPSExecutorSharedMemoryMapperServiceInitializeSignature =
+    shared::SPSExpected<shared::SPSExecutorAddr>(shared::SPSExecutorAddr,
+                                                 shared::SPSExecutorAddr,
+                                                 shared::SPSFinalizeRequest);
+using SPSExecutorSharedMemoryMapperServiceDeinitializeSignature =
+    shared::SPSError(shared::SPSExecutorAddr,
+                     shared::SPSSequence<shared::SPSExecutorAddr>);
+using SPSExecutorSharedMemoryMapperServiceReleaseSignature = shared::SPSError(
+    shared::SPSExecutorAddr, shared::SPSSequence<shared::SPSExecutorAddr>);
+
 using SPSRunAsMainSignature = int64_t(shared::SPSExecutorAddr,
                                       shared::SPSSequence<shared::SPSString>);
 
 } // end namespace rt
 } // end namespace orc
 } // end namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_ORC_SHARED_ORCRTBRIDGE_H

llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.h

+//===----------- ExecutorSharedMemoryMapperService.h ------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_EXECUTORSHAREDMEMORYMAPPERSERVICE
+#define LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_EXECUTORSHAREDMEMORYMAPPERSERVICE
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h"
+#include "llvm/ExecutionEngine/Orc/TargetProcess/ExecutorBootstrapService.h"
+
+#include <atomic>
+#include <mutex>
+
+namespace llvm {
+namespace orc {
+namespace rt_bootstrap {
+
+class ExecutorSharedMemoryMapperService final
+    : public ExecutorBootstrapService {
+public:
+  ~ExecutorSharedMemoryMapperService(){};
+
+  Expected<std::pair<ExecutorAddr, std::string>> reserve(uint64_t Size);
+  Expected<ExecutorAddr> initialize(ExecutorAddr Reservation,
+                                    tpctypes::FinalizeRequest &FR);
+
+  Error deinitialize(const std::vector<ExecutorAddr> &Bases);
+  Error release(const std::vector<ExecutorAddr> &Bases);
+
+  Error shutdown() override;
+  void addBootstrapSymbols(StringMap<ExecutorAddr> &M) override;
+
+private:
+  struct Allocation {
+    std::vector<shared::WrapperFunctionCall> DeinitializationActions;
+  };
+  using AllocationMap = DenseMap<ExecutorAddr, Allocation>;
+
+  struct Reservation {
+    size_t Size;
+    std::vector<ExecutorAddr> Allocations;
+  };
+  using ReservationMap = DenseMap<void *, Reservation>;
+
+  Error releaseImpl(void *Base, Allocation &A);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  reserveWrapper(const char *ArgData, size_t ArgSize);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  initializeWrapper(const char *ArgData, size_t ArgSize);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  deinitializeWrapper(const char *ArgData, size_t ArgSize);
+
+  static llvm::orc::shared::CWrapperFunctionResult
+  releaseWrapper(const char *ArgData, size_t ArgSize);
+
+  std::atomic<int> SharedMemoryCount{0};
+  std::mutex Mutex;
+  ReservationMap Reservations;
+  AllocationMap Allocations;
+};
+
+} // namespace rt_bootstrap
+} // namespace orc
+} // namespace llvm
+#endif // LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_EXECUTORSHAREDMEMORYMAPPERSERVICE

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

 //===- MemoryMapper.cpp - Cross-process memory mapper ------------*- C++ -*-==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ExecutionEngine/Orc/MemoryMapper.h"
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+
+#if defined(LLVM_ON_UNIX)
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#endif
 
 namespace llvm {
 namespace orc {
 
 MemoryMapper::~MemoryMapper() {}
 
 void InProcessMemoryMapper::reserve(size_t NumBytes,
                                     OnReservedFunction OnReserved) {
InProcessMemoryMapper::~InProcessMemoryMapper() {
   }
 
   std::promise<MSVCPError> P;
   auto F = P.get_future();
   release(ReservationAddrs, [&](Error Err) { P.set_value(std::move(Err)); });
   cantFail(F.get());
 }
 
+// SharedMemoryMapper
+
+void SharedMemoryMapper::reserve(size_t NumBytes,
+                                 OnReservedFunction OnReserved) {
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceReserveSignature>(
+      SAs.Reserve,
+      [this, NumBytes, OnReserved = std::move(OnReserved)](
+          Error SerializationErr,
+          Expected<std::pair<ExecutorAddr, std::string>> Result) mutable {
+        if (SerializationErr) {
+          cantFail(Result.takeError());
+          return OnReserved(std::move(SerializationErr));
+        }
+
+        if (!Result)
+          return OnReserved(Result.takeError());
+
+        ExecutorAddr RemoteAddr;
+        std::string SharedMemoryName;
+        std::tie(RemoteAddr, SharedMemoryName) = std::move(*Result);
+
+#if defined(LLVM_ON_UNIX)
+        int SharedMemoryFile = shm_open(SharedMemoryName.c_str(), O_RDWR, 0700);
+        if (SharedMemoryFile < 0) {
+          return OnReserved(errorCodeToError(
+              std::error_code(errno, std::generic_category())));
+        }
+
+        // this prevents other processes from accessing it by name
+        shm_unlink(SharedMemoryName.c_str());
+
+        void *LocalAddr = mmap(nullptr, NumBytes, PROT_READ | PROT_WRITE,
+                               MAP_SHARED, SharedMemoryFile, 0);
+        if (LocalAddr == MAP_FAILED) {
+          return OnReserved(errorCodeToError(
+              std::error_code(errno, std::generic_category())));
+        }
+
+        close(SharedMemoryFile);
+#else
+        llvm_unreachable();
+#endif
+        {
+          std::lock_guard<std::mutex> Lock(Mutex);
+          Reservations.insert({RemoteAddr, {LocalAddr, NumBytes}});
+        }
+
+        OnReserved(ExecutorAddrRange(RemoteAddr, NumBytes));
+      },
+      SAs.Instance, NumBytes);
+}
+
+char *SharedMemoryMapper::prepare(ExecutorAddr Addr, size_t ContentSize) {
+  auto R = Reservations.upper_bound(Addr);
+  assert(R != Reservations.begin() && "Attempt to prepare unknown range");
+  R--;
+
+  ExecutorAddrDiff Offset = Addr - R->first;
+
+  return static_cast<char *>(R->second.LocalAddr) + Offset;
+}
+
+void SharedMemoryMapper::initialize(MemoryMapper::AllocInfo &AI,
+                                    OnInitializedFunction OnInitialized) {
+  auto Reservation = Reservations.find(AI.MappingBase);
+  assert(Reservation != Reservations.end() &&
+         "Attempt to initialize unreserved range");
+
+  tpctypes::FinalizeRequest FR;
+
+  AI.Actions.swap(FR.Actions);
+
+  FR.Segments.reserve(AI.Segments.size());
+
+  for (auto Segment : AI.Segments) {
+    char *Base =
+        static_cast<char *>(Reservation->second.LocalAddr) + Segment.Offset;
+    std::memset(Base + Segment.ContentSize, 0, Segment.ZeroFillSize);
+
+    tpctypes::SegFinalizeRequest SegReq;
+    SegReq.Prot = tpctypes::toWireProtectionFlags(
+        static_cast<sys::Memory::ProtectionFlags>(Segment.Prot));
+    SegReq.Addr = AI.MappingBase + Segment.Offset;
+    SegReq.Size = Segment.ContentSize + Segment.ZeroFillSize;
+
+    FR.Segments.push_back(SegReq);
+  }
+
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceInitializeSignature>(
+      SAs.Initialize,
+      [OnInitialized = std::move(OnInitialized)](
+          Error SerializationErr, Expected<ExecutorAddr> Result) mutable {
+        if (SerializationErr) {
+          cantFail(Result.takeError());
+          return OnInitialized(std::move(SerializationErr));
+        }
+
+        OnInitialized(std::move(Result));
+      },
+      SAs.Instance, AI.MappingBase, std::move(FR));
+}
+
+void SharedMemoryMapper::deinitialize(
+    ArrayRef<ExecutorAddr> Allocations,
+    MemoryMapper::OnDeinitializedFunction OnDeinitialized) {
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceDeinitializeSignature>(
+      SAs.Deinitialize,
+      [OnDeinitialized = std::move(OnDeinitialized)](Error SerializationErr,
+                                                     Error Result) mutable {
+        if (SerializationErr) {
+          cantFail(std::move(Result));
+          return OnDeinitialized(std::move(SerializationErr));
+        }
+
+        OnDeinitialized(std::move(Result));
+      },
+      SAs.Instance, Allocations);
+}
+
+void SharedMemoryMapper::release(ArrayRef<ExecutorAddr> Bases,
+                                 OnReleasedFunction OnReleased) {
+  Error Err = Error::success();
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+
+    for (auto Base : Bases) {
+#if defined(LLVM_ON_UNIX)
+      if (munmap(Base.toPtr<void *>(), Reservations[Base].Size) != 0)
+        Err = joinErrors(std::move(Err), errorCodeToError(std::error_code(
+                                             errno, std::generic_category())));
+#else
+      llvm_unreachable();
+#endif
+
+      Reservations.erase(Base);
+    }
+  }
+
+  EPC.callSPSWrapperAsync<
+      rt::SPSExecutorSharedMemoryMapperServiceReleaseSignature>(
+      SAs.Release,
+      [OnReleased = std::move(OnReleased),
+       Err = std::move(Err)](Error SerializationErr, Error Result) mutable {
+        if (SerializationErr) {
+          cantFail(std::move(Result));
+          return OnReleased(
+              joinErrors(std::move(Err), std::move(SerializationErr)));
+        }
+
+        return OnReleased(joinErrors(std::move(Err), std::move(Result)));
+      },
+      SAs.Instance, Bases);
+}
+
+SharedMemoryMapper::~SharedMemoryMapper() {
+  std::vector<ExecutorAddr> ReservationAddrs;
+  if (!Reservations.empty()) {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    {
+      ReservationAddrs.reserve(Reservations.size());
+      for (const auto &R : Reservations) {
+        ReservationAddrs.push_back(R.first);
+      }
+    }
+  }
+
+  std::promise<MSVCPError> P;
+  auto F = P.get_future();
+  release(ReservationAddrs, [&](Error Err) { P.set_value(std::move(Err)); });
+  cantFail(F.get());
+}
+
 } // namespace orc
 
 } // namespace llvm

llvm/lib/ExecutionEngine/Orc/Shared/OrcRTBridge.cpp

 namespace rt {
 
 const char *SimpleExecutorDylibManagerInstanceName =
     "__llvm_orc_SimpleExecutorDylibManager_Instance";
 const char *SimpleExecutorDylibManagerOpenWrapperName =
     "__llvm_orc_SimpleExecutorDylibManager_open_wrapper";
 const char *SimpleExecutorDylibManagerLookupWrapperName =
     "__llvm_orc_SimpleExecutorDylibManager_lookup_wrapper";
+
 const char *SimpleExecutorMemoryManagerInstanceName =
     "__llvm_orc_SimpleExecutorMemoryManager_Instance";
 const char *SimpleExecutorMemoryManagerReserveWrapperName =
     "__llvm_orc_SimpleExecutorMemoryManager_reserve_wrapper";
 const char *SimpleExecutorMemoryManagerFinalizeWrapperName =
     "__llvm_orc_SimpleExecutorMemoryManager_finalize_wrapper";
 const char *SimpleExecutorMemoryManagerDeallocateWrapperName =
     "__llvm_orc_SimpleExecutorMemoryManager_deallocate_wrapper";
+
+const char *ExecutorSharedMemoryMapperServiceInstanceName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Instance";
+const char *ExecutorSharedMemoryMapperServiceReserveWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Reserve";
+const char *ExecutorSharedMemoryMapperServiceInitializeWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Initialize";
+const char *ExecutorSharedMemoryMapperServiceDeinitializeWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Deinitialize";
+const char *ExecutorSharedMemoryMapperServiceReleaseWrapperName =
+    "__llvm_orc_ExecutorSharedMemoryMapperService_Release";
+
 const char *MemoryWriteUInt8sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint8s_wrapper";
 const char *MemoryWriteUInt16sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint16s_wrapper";
 const char *MemoryWriteUInt32sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint32s_wrapper";
 const char *MemoryWriteUInt64sWrapperName =
     "__llvm_orc_bootstrap_mem_write_uint64s_wrapper";
 const char *MemoryWriteBuffersWrapperName =
     "__llvm_orc_bootstrap_mem_write_buffers_wrapper";
+
 const char *RegisterEHFrameSectionWrapperName =
     "__llvm_orc_bootstrap_register_ehframe_section_wrapper";
 const char *DeregisterEHFrameSectionWrapperName =
     "__llvm_orc_bootstrap_deregister_ehframe_section_wrapper";
+
 const char *RunAsMainWrapperName = "__llvm_orc_bootstrap_run_as_main_wrapper";
 
 } // end namespace rt
 } // end namespace orc
 } // end namespace llvm

llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt

 add_llvm_component_library(LLVMOrcTargetProcess
+  ExecutorSharedMemoryMapperService.cpp
   JITLoaderGDB.cpp
   OrcRTBootstrap.cpp
   RegisterEHFrames.cpp
   SimpleExecutorDylibManager.cpp
   SimpleExecutorMemoryManager.cpp
   SimpleRemoteEPCServer.cpp
   TargetExecutionUtils.cpp
 

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

+//===---------- ExecutorSharedMemoryMapperService.cpp -----------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.h"
+
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+#include "llvm/Support/Process.h"
+
+#include <sstream>
+
+#ifdef LLVM_ON_UNIX
+#include <errno.h>
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#endif
+
+namespace llvm {
+namespace orc {
+namespace rt_bootstrap {
+
+Expected<std::pair<ExecutorAddr, std::string>>
+ExecutorSharedMemoryMapperService::reserve(uint64_t Size) {
+#if defined(LLVM_ON_UNIX)
+  std::string SharedMemoryName;
+  {
+    std::stringstream SharedMemoryNameStream;
+    SharedMemoryNameStream << "/jitlink_" << sys::Process::getProcessId() << '_'
+                           << (++SharedMemoryCount);
+    SharedMemoryName = SharedMemoryNameStream.str();
+  }
+
+  int SharedMemoryFile =
+      shm_open(SharedMemoryName.c_str(), O_RDWR | O_CREAT | O_EXCL, 0700);
+  if (SharedMemoryFile < 0) {
+    return errorCodeToError(std::error_code(errno, std::generic_category()));
+  }
+
+  // by default size is 0
+  if (ftruncate(SharedMemoryFile, Size) < 0) {
+    return errorCodeToError(std::error_code(errno, std::generic_category()));
+  }
+
+  void *Addr = mmap(nullptr, Size, PROT_NONE, MAP_SHARED, SharedMemoryFile, 0);
+  if (Addr == MAP_FAILED) {
+    return errorCodeToError(std::error_code(errno, std::generic_category()));
+  }
+
+  close(SharedMemoryFile);
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    Reservations[Addr].Size = Size;
+  }
+
+  return std::make_pair(ExecutorAddr::fromPtr(Addr),
+                        std::move(SharedMemoryName));
+#else
+  llvm_unreachable();
+#endif
+}
+
+Expected<ExecutorAddr>
+ExecutorSharedMemoryMapperService::initialize(ExecutorAddr Reservation,
+                                              tpctypes::FinalizeRequest &FR) {
+  ExecutorAddr MinAddr(~0ULL);
+
+  // Contents are already in place
+  for (auto &Segment : FR.Segments) {
+    if (Segment.Addr < MinAddr)
+      MinAddr = Segment.Addr;
+
+#if defined(LLVM_ON_UNIX)
+    int NativeProt = 0;
+    if (Segment.Prot & tpctypes::WPF_Read)
+      NativeProt |= PROT_READ;
+    if (Segment.Prot & tpctypes::WPF_Write)
+      NativeProt |= PROT_WRITE;
+    if (Segment.Prot & tpctypes::WPF_Exec)
+      NativeProt |= PROT_EXEC;
+
+    mprotect(Segment.Addr.toPtr<void *>(), Segment.Size, NativeProt);
+
+    if (Segment.Prot & tpctypes::WPF_Exec)
+      sys::Memory::InvalidateInstructionCache(Segment.Addr.toPtr<void *>(),
+                                              Segment.Size);
+  }
+#else
+    llvm_unreachable();
+#endif
+
+  // Run finalization actions and get deinitlization action list.
+  auto DeinitializeActions = shared::runFinalizeActions(FR.Actions);
+  if (!DeinitializeActions) {
+    return DeinitializeActions.takeError();
+  }
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    Allocations[MinAddr].DeinitializationActions =
+        std::move(*DeinitializeActions);
+    Reservations[Reservation.toPtr<void *>()].Allocations.push_back(MinAddr);
+  }
+
+  return MinAddr;
+}
+
+Error ExecutorSharedMemoryMapperService::deinitialize(
+    const std::vector<ExecutorAddr> &Bases) {
+  Error AllErr = Error::success();
+
+  {
+    std::lock_guard<std::mutex> Lock(Mutex);
+
+    for (auto Base : Bases) {
+      if (Error Err = shared::runDeallocActions(
+              Allocations[Base].DeinitializationActions)) {
+        AllErr = joinErrors(std::move(AllErr), std::move(Err));
+      }
+
+      Allocations.erase(Base);
+    }
+  }
+
+  return AllErr;
+}
+
+Error ExecutorSharedMemoryMapperService::release(
+    const std::vector<ExecutorAddr> &Bases) {
+  Error Err = Error::success();
+
+  for (auto Base : Bases) {
+    std::vector<ExecutorAddr> AllocAddrs;
+    size_t Size;
+    {
+      std::lock_guard<std::mutex> Lock(Mutex);
+      auto &R = Reservations[Base.toPtr<void *>()];
+      Size = R.Size;
+      AllocAddrs.swap(R.Allocations);
+    }
+
+    // deinitialize sub allocations
+    if (Error E = deinitialize(AllocAddrs))
+      Err = joinErrors(std::move(Err), std::move(E));
+
+#if defined(LLVM_ON_UNIX)
+    if (munmap(Base.toPtr<void *>(), Size) != 0)
+      Err = joinErrors(std::move(Err), errorCodeToError(std::error_code(
+                                           errno, std::generic_category())));
+#else
+      llvm_unreachable();
+#endif
+
+    std::lock_guard<std::mutex> Lock(Mutex);
+    Reservations.erase(Base.toPtr<void *>());
+  }
+
+  return Err;
+}
+
+Error ExecutorSharedMemoryMapperService::shutdown() {
+  std::vector<ExecutorAddr> ReservationAddrs;
+  if (!Reservations.empty()) {
+    std::lock_guard<std::mutex> Lock(Mutex);
+    {
+      ReservationAddrs.reserve(Reservations.size());
+      for (const auto &R : Reservations) {
+        ReservationAddrs.push_back(ExecutorAddr::fromPtr(R.getFirst()));
+      }
+    }
+  }
+  return release(ReservationAddrs);
+
+  return Error::success();
+}
+
+void ExecutorSharedMemoryMapperService::addBootstrapSymbols(
+    StringMap<ExecutorAddr> &M) {
+  M[rt::ExecutorSharedMemoryMapperServiceInstanceName] =
+      ExecutorAddr::fromPtr(this);
+  M[rt::ExecutorSharedMemoryMapperServiceReserveWrapperName] =
+      ExecutorAddr::fromPtr(&reserveWrapper);
+  M[rt::ExecutorSharedMemoryMapperServiceInitializeWrapperName] =
+      ExecutorAddr::fromPtr(&initializeWrapper);
+  M[rt::ExecutorSharedMemoryMapperServiceDeinitializeWrapperName] =
+      ExecutorAddr::fromPtr(&deinitializeWrapper);
+  M[rt::ExecutorSharedMemoryMapperServiceReleaseWrapperName] =
+      ExecutorAddr::fromPtr(&releaseWrapper);
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::reserveWrapper(const char *ArgData,
+                                                  size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceReserveSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::reserve))
+          .release();
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::initializeWrapper(const char *ArgData,
+                                                     size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceInitializeSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::initialize))
+          .release();
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::deinitializeWrapper(const char *ArgData,
+                                                       size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceDeinitializeSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::deinitialize))
+          .release();
+}
+
+llvm::orc::shared::CWrapperFunctionResult
+ExecutorSharedMemoryMapperService::releaseWrapper(const char *ArgData,
+                                                  size_t ArgSize) {
+  return shared::WrapperFunction<
+             rt::SPSExecutorSharedMemoryMapperServiceReleaseSignature>::
+      handle(ArgData, ArgSize,
+             shared::makeMethodWrapperHandler(
+                 &ExecutorSharedMemoryMapperService::release))
+          .release();
+}
+
+} // namespace rt_bootstrap
+} // end namespace orc
+} // end namespace llvm

llvm/unittests/ExecutionEngine/Orc/CMakeLists.txt

add_llvm_unittest(OrcJITTests
   LazyCallThroughAndReexportsTest.cpp
   LookupAndRecordAddrsTest.cpp
   MemoryMapperTest.cpp
   ObjectLinkingLayerTest.cpp
   OrcCAPITest.cpp
   OrcTestCommon.cpp
   ResourceTrackerTest.cpp
   RTDyldObjectLinkingLayerTest.cpp
+  SharedMemoryMapperTest.cpp
   SimpleExecutorMemoryManagerTest.cpp
   SimplePackedSerializationTest.cpp
   SymbolStringPoolTest.cpp
   TaskDispatchTest.cpp
   ThreadSafeModuleTest.cpp
   WrapperFunctionUtilsTest.cpp
   )
 
 target_link_libraries(OrcJITTests PRIVATE
                         LLVMTestingSupport
                         ${ORC_JIT_TEST_LIBS})
 
 set_property(TARGET OrcJITTests PROPERTY FOLDER "Tests/UnitTests/ExecutionTests")

llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp

+//===- SharedMemoryMapperTest.cpp -- Tests for SharedMemoryMapper ---------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "OrcTestCommon.h"
+
+#include "llvm/ExecutionEngine/Orc/MemoryMapper.h"
+#include "llvm/ExecutionEngine/Orc/Shared/OrcRTBridge.h"
+#include "llvm/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.h"
+#include "llvm/Testing/Support/Error.h"
+
+using namespace llvm;
+using namespace llvm::orc;
+using namespace llvm::orc::shared;
+using namespace llvm::orc::rt_bootstrap;
+
+// A basic function to be used as both initializer/deinitializer
+orc::shared::CWrapperFunctionResult incrementWrapper(const char *ArgData,
+                                                     size_t ArgSize) {
+  return WrapperFunction<SPSError(SPSExecutorAddr)>::handle(
+             ArgData, ArgSize,
+             [](ExecutorAddr A) -> Error {
+               *A.toPtr<int *>() += 1;
+               return Error::success();
+             })
+      .release();
+}
+
+TEST(SharedMemoryMapperTest, MemReserveInitializeDeinitializeRelease) {
+  // These counters are used to track how many times the initializer and
+  // deinitializer functions are called
+  int InitializeCounter = 0;
+  int DeinitializeCounter = 0;
+
+  auto SelfEPC = cantFail(SelfExecutorProcessControl::Create());
+
+  ExecutorSharedMemoryMapperService MapperService;
+
+  SharedMemoryMapper::SymbolAddrs SAs;
+  {
+    StringMap<ExecutorAddr> Map;
+    MapperService.addBootstrapSymbols(Map);
+    SAs.Instance = Map[rt::ExecutorSharedMemoryMapperServiceInstanceName];
+    SAs.Reserve = Map[rt::ExecutorSharedMemoryMapperServiceReserveWrapperName];
+    SAs.Initialize =
+        Map[rt::ExecutorSharedMemoryMapperServiceInitializeWrapperName];
+    SAs.Deinitialize =
+        Map[rt::ExecutorSharedMemoryMapperServiceDeinitializeWrapperName];
+    SAs.Release = Map[rt::ExecutorSharedMemoryMapperServiceReleaseWrapperName];
+  }
+
+  std::string TestString = "Hello, World!";
+
+  // barrier
+  std::promise<void> P;
+  auto F = P.get_future();
+
+  auto PageSize = cantFail(sys::Process::getPageSize());
+  size_t ReqSize = PageSize;
+  {
+    std::unique_ptr<MemoryMapper> Mapper =
+        std::make_unique<SharedMemoryMapper>(*SelfEPC, SAs);
+
+    Mapper->reserve(ReqSize, [&](Expected<ExecutorAddrRange> Result) {
+      EXPECT_THAT_ERROR(Result.takeError(), Succeeded());
+      auto Reservation = std::move(*Result);
+      {
+        char *Addr = Mapper->prepare(Reservation.Start, TestString.size() + 1);
+        std::strcpy(Addr, TestString.c_str());
+      }
+      MemoryMapper::AllocInfo AI;
+      {
+        MemoryMapper::AllocInfo::SegInfo SI;
+        SI.Offset = 0;
+        SI.ContentSize = TestString.size() + 1;
+        SI.ZeroFillSize = PageSize - SI.ContentSize;
+        SI.Prot = sys::Memory::MF_READ | sys::Memory::MF_WRITE;
+
+        AI.MappingBase = Reservation.Start;
+        AI.Segments.push_back(SI);
+        AI.Actions.push_back(
+            {cantFail(WrapperFunctionCall::Create<SPSArgList<SPSExecutorAddr>>(
+                 ExecutorAddr::fromPtr(incrementWrapper),
+                 ExecutorAddr::fromPtr(&InitializeCounter))),
+             cantFail(WrapperFunctionCall::Create<SPSArgList<SPSExecutorAddr>>(
+                 ExecutorAddr::fromPtr(incrementWrapper),
+                 ExecutorAddr::fromPtr(&DeinitializeCounter)))});
+      }
+
+      EXPECT_EQ(InitializeCounter, 0);
+      EXPECT_EQ(DeinitializeCounter, 0);
+
+      Mapper->initialize(AI, [&](Expected<ExecutorAddr> Result) {
+        EXPECT_THAT_ERROR(Result.takeError(), Succeeded());
+
+        EXPECT_EQ(TestString, std::string(static_cast<char *>(
+                                  Reservation.Start.toPtr<char *>())));
+
+        EXPECT_EQ(InitializeCounter, 1);
+        EXPECT_EQ(DeinitializeCounter, 0);
+
+        Mapper->deinitialize({*Result}, [&](Error Err) {
+          EXPECT_THAT_ERROR(std::move(Err), Succeeded());
+
+          EXPECT_EQ(InitializeCounter, 1);
+          EXPECT_EQ(DeinitializeCounter, 1);
+
+          Mapper->release({Reservation.Start}, [&](Error Err) {
+            EXPECT_THAT_ERROR(std::move(Err), Succeeded());
+
+            P.set_value();
+          });
+        });
+      });
+    });
+
+    F.wait();
+    // Mapper must be destructed before calling shutdown to avoid double free
+  }
+
+  EXPECT_THAT_ERROR(MapperService.shutdown(), Succeeded());
+  cantFail(SelfEPC->disconnect());
+}