[SystemZ][z/OS] Add ASCII and 32-bit variants for libc++.

[zibi]

Summary

This patch enables libc++ build as shared library in all combinations of ASCII/EBCDIC and 32-bit/64-bit variants. In particular it introduces:

1. ASCII version of libc++ named as libc++_a.so
2. Script to rename DLL name inside the generated side deck
3. Various names for dataset members where DLL libraries and their side decks will reside
4. Add the following options:
   • LIBCXX_SHARED_OUTPUT_NAME
   • LIBCXX_ADDITIONAL_COMPILE_FLAGS
   • LIBCXX_ADDITIONAL_LIBRARIES
   • LIBCXXABI_ADDITIONAL_COMPILE_FLAGS
   • LIBCXXABI_ADDITIONAL_LIBRARIES

Background and rational of this patch

The linker on z/OS creates a list of exported symbols in a file called side deck. The list contains the symbol name as well as the name of the DLL which implements the symbol. The name of the DLL depends on what is specified in the -o command line option. If it points to a USS file, than the DLL name in the side deck will be the USS file name. If it points to a member of a dataset then the DLL name in the side deck is the member name.

If CMake could deal with z/OS datasets we could use -o that points to a dataset member name, but this does not seem to work so we have to produce a USS file as the DLL and then copy the content of the produced side deck to a dataset as well as rename the USS file name in the side deck to a dataset member name that corresponds to that DLL.

[zibi]

ping

[muiez]

LGTM

[zibi]

@libc++ ping

[zibi]

Rebase with latest and removing conflicts.

[SeanP]

We'll have to create another patch to apply the side deck renaming to the libc++exception added in https://reviews.llvm.org/D118620/new/.

Otherwise LGTM

[zibi]

NFC: just trying to clean CI status

[zibi]

ping

[zibi]

This comment was removed by zibi.

[muiez]

libcxx/src/CMakeLists.txt

216–219

I don't think we need this line since the COMPILE_FLAGS is already set for that target (line 218)

[muiez]

libcxx/src/CMakeLists.txt

223

We don't need to set ZOS_BUILD_32_BITS here anymore, as it is not used.

[muiez]

libcxx/src/CMakeLists.txt

221

The LIBCXX_TARGET_TRIPLE is deprecated.

[ldionne]

I can't support this patch as-is. It upstreams vendor-specific quirks for a single platform, causing the generic upstream code to be side-by-side with platform-specific code for z/OS only. Please try to put yourself in the shoes of the open-source community that doesn't work on z/OS on a day to day basis: this patch basically adds a bunch of platform-specific complexity (i.e. tech debt) that everyone will have to deal with, for essentially only z/OS's benefit. This is not acceptable.

Take for instance the example of Fuchsia, another platform that uses libc++. @phosek added support for a bunch of stuff to support Fuchsia's build, for example LIBCXX_HERMETIC_STATIC_LIBRARY & friends, or LIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY. Even though I'm not a fan of those because they increase complexity, they are generic and one could argue they are generally useful, so it makes sense to try and support them upstream. I am not sure whether Fuchsia is using the CMake build at this point (I don't think it does yet), however my point is this: how many references to Fuchsia specific stuff can you count in our CMakeLists.txt? Very few, because they found ways to implement most of what they needed generically.

Similarly, at Apple, we used to build libc++ and libc++abi with a Xcode project a couple of years ago. I did a bunch of work to migrate that build over to CMake, and in doing so, I had to introduce a bunch of Apple-specific tweaks to the CMake build. I have been upstreaming things that make sense one tiny bit at a time, but even three years later, we are still living with a downstream diff that I don't think would make sense for upstream to support, so I'm keeping it downstream until I can find reasonable ways to upstream it all. It would have been much easier to just upstream the whole thing from the start, however LLVM is not a dumping ground for vendors, so it would have been bad open-source citizenship to do that.

So:

• Libc++ welcomes contributions for adding support to new platforms -- it makes it more widely useful, which is awesome.
• Libc++ is not a dumping ground for arbitrary vendor-specific stuff -- there's too many vendors with too many use cases that require tweaking libc++, we couldn't realistically accommodate everyone even if we wanted to.
• Having some amount of downstream-only diff is normal. The key is to manage it to reduce downstream merge conflicts.

Also, the devil's in the details -- I am much less concerned by adding something self-contained like zos_rename_dll_side_deck.sh than tweaking the CMake itself, since we can basically forget about zos_rename_dll_side_deck.sh without harm, however we'll have to deal with the CMake complexity forever. Of course, upstreaming zos_rename_dll_side_deck.sh on its own doesn't make much sense -- I'm just trying to nuance my position.

FWIW, I think there might be ways to achieve some of what you want in a way that benefits everyone. Several vendors need to run post-install scripts or have slightly more complicated ways of building the library (see for example libcxx/utils/ci/apple-install-libcxx.sh, which I consider terrible tech debt for upstream and want to burn with fire). I suspect it may be possible to add customization points to the CMake build in a way that vendors can more easily configure it without adding tech debt for everyone, kind of like what we did for new style testing configurations (see for example libcxx/test/configs/apple-libc++-backdeployment.cfg.in).

libcxx/include/__config_site.in

13 ↗

(On Diff #409265)

We don't put any logic inside __config_site. This logic should instead be implemented by setting a different namespace in the cache you use when building for EBCDIC/ASCII (you probably want two caches).

libcxx/utils/libcxx/test/config.py

377 ↗

(On Diff #409265)

You should be using a new style config (see libcxx/test/configs/), not the legacy system. This is much easier to tweak for your exact needs.

[muiez]

libcxx/utils/libcxx/test/config.py

377–378 ↗

(On Diff #409265)

We can avoid this, since we moved to the new config file. Will create a separate patch for this.

[SeanP]

libcxx/include/__config_site.in

13 ↗

(On Diff #409265)

Unless we are missing something, I don't think this kind of requirement is reasonable. We need to be able to the distinguish between ebcdic/ascii with the installed headers. If we don't put conditional logic into the __config header, we will be creating horribly complicated code (if it's even possible) in the compiler. If we can't put conditional logic into __config, we will be required to ship two copies of the the libc++ headers (one where __config is set up for ascii and one where it is set up for ebcdic). This is going to force the compiler to change the include dir for libc++ based on the the ascii/ebcdic option. That is going to break the -isystem option.

We need one set of header files that defines _LIBCPP_ABI_NAMESPACE differently at compile time depending on if it is ascii or ebcdic. This is the only way we can see doing this. If there is a better way we are open to doing it that way. The cache files don't work as they would result in two different set of headers.

[SeanP]

libcxx/src/CMakeLists.txt

380

I'd either wrap all of these install commands in one if (not ZOS) or get rid of them completely.

[SeanP]

libcxx/src/CMakeLists.txt

380

Just to be clear, I meant get rid of the "NOT ZOS AND" not the install commands.

[zibi]

This patch addresses the comments from Sean and Muiez. It reduces the original patch and makes it up-to-date.

[zibi]

The side deck was created in ASCII mode which is not desired. This revision corrects that.

[zibi]

• Rebasing and resolving conflicts

No functional changes are introduced since the last patch.

[zibi]

The CI is clean now, Can I have the approval or more request, Thx.

[zibi]

Rebase to pick up 0d7859765260e1dddfdc3490c1fe35537775bb41 which hopefully will fix CL.

[zibi]

@ldionne Louis, all comments have been addressed CI is green, can you look at it again? Thx.

[zibi]

ping, still waiting for the review ...

[ldionne]

I left some comments. This still doesn't meet the bar since it hardcodes a bunch of z/OS specific knowledge in the build. However, I can see ways out of it, as explained in the comments.

In D118503#3646431, @zibi wrote:

The CI is clean now, Can I have the approval or more request, Thx.

AFAICT we don't have z/OS CI set up anyway, so it wouldn't show up even if the patch didn't work. At least we do know it does not break other configurations, indeed.

libcxx/include/__config_site.in

13 ↗

(On Diff #409265)

If we don't put conditional logic into the __config header

Logic inside __config is fine, but logic inside __config_site is not. This is adding logic inside __config_site.

We made changes to Clang to make it easy to ship libc++ headers as two things:

1. A per-target include directory that contains __config_site (and only that)
2. A target-agnostic c++/v1 directory that contains all the other headers

Then, the driver selects the right target-specific directory so the right __config_site header is picked up. I believe you should be using that instead. If you did, it would actually be quite nice: there wouldn't be any specific logic here since you'd define a different _LIBCPP_ABI_NAMESPACE in each build of libc++ that you do (ASCII and non-ASCII), and that would result in two different __config_sites in different directories.

libcxx/src/CMakeLists.txt

289

The only things that this target does differently from cxx_shared is pass -fzos-le-char-mode=ascii, set the output name to c++_a and add a post-build command.

Instead, the following would avoid hardcoding platform-specific knowledge in our build:

1. Build twice, with CMAKE_CXX_COMPILE_FLAGS=-fzos-le-char-mode=ascii|ebcdic. This would also remove the need for setting add_target_flags_if_supported("-fzos-le-char-mode=ebcdic") in libcxxabi/CMakeLists.txt
2. Add a way to customize the output name of libc++ (by default it would be c++, but you'd set it to c++_a in your CMake cache)
3. Add your post-build step to cxx_shared under some option. This isn't clean, but I could refactor that eventually to provide a way to run arbitrary post-build steps more cleanly.

[zibi]

Louis, thank you for the review and providing some suggestions. However, I'm still not sure how to proceed see my inline comments.

libcxx/CMakeLists.txt

466–467

The add_target_flags_if_supported("-fzos-le-char-mode=ebcdic") will be removed since I found an alternative way.

libcxx/include/__config_site.in

13 ↗

(On Diff #409265)

I'm not entire sure how you envision this. You can not move the logic to __config since #cmakedefine has to be preprocessed by cmake. More importantly I think you asking for 2 separate builds which we want to avoid. See my other inline comment.

libcxx/src/CMakeLists.txt

289

Does current build infrastructure allow to reuse the previous build and build only c++?

We build c++ only in both encoding modes and the rest of the libraries in EBCDIC. The c++ has dependency on rest of the libraries like c++_abi, unwinder, etc. We did not want to duplicated the entire library in both modes.

The first build would be the EBCDIC one and that is all good. However, the second ASCII build would throw away anything but c++_a. The most importantly issue would be how to make dependency to EBCDIC libraries from the 1st build rather then ASCII build from the 2nd one.

libcxxabi/CMakeLists.txt

244–247

The add_target_flags_if_supported("-fzos-le-char-mode=ebcdic") will be removed since I found an alternative way.

[zibi]

Addressing Louis's comments to hide z/OS changes as much as possible in cache.

[ldionne]

libcxx/CMakeLists.txt

467–469

Instead, let's add an option like LIBCXX_ADDITIONAL_COMPILE_FLAGS that we can use to pass those compiler flags from the cache. I'm not a huge fan of doing that, but I do think it's a useful backdoor to have.

However, even with LIBCXX_ADDITIONAL_COMPILE_FLAGS, I don't understand why -I../${CMAKE_CXX_COMPILER_TARGET}/${LIBCXX_INSTALL_INCLUDE_DIR} is needed. What purpose does it serve?

libcxx/cmake/caches/s390x-ibm-zos-ascii.cmake

16–17

Why are we setting LLVM_FOO variables in the CMake cache for libc++? Unless I am mistaken, I think neither of those two cache values will be picked up by anything in the runtimes build, and so they serve no purpose AFAICT.

libcxx/cmake/caches/s390x-ibm-zos.cmake

26

Same here, this does not exist.

libcxx/cmake/caches/s390x32-ibm-zos-ascii.cmake

23

I don't see this used anywhere. Should this be defined in CMakeLists.txt and then overriden here only with set(LIBCXX_SHARED_OUTPUT_NAME "c++_a")?

25

This is not used anywhere. Is there a reason why it's in all the caches?

libcxx/src/CMakeLists.txt

203–204

Why do you avoid linking cxx_shared against its dependencies in the z/OS build? Is it because you disable building the headers? I don't think this should be z/OS specific, this should probably be supported out of the box instead.

[zibi]

answering comments and some questions

libcxx/CMakeLists.txt

467–469

The -I.. is needed because we build only cxx_shared as ASCII and need to point to headers cxx-headers and cxxabi-headers from the full build we do in EBCDIC.

libcxx/cmake/caches/s390x-ibm-zos-ascii.cmake

16–17

Yes, you are right I will remove them.

libcxx/cmake/caches/s390x32-ibm-zos-ascii.cmake

23

Yes, this is how I have implemented it in my local branch. For some reason it was not included in this patch. I will correct this.

25

This is needed for the exception library which we created by splitting cxx which is not included in this patch but it will be included in separate patch. The reason I'm keeping it here so I can easy verify current patch on my local branch.

libcxx/src/CMakeLists.txt

203–204

Yes we disable everything including headers when building cxx_shared in ASCII and want to link with dependencies from the full separate build done in EBCDIC. This is why we need -I and -Wl,<side-deck>.x options.

What do you mean out of the box. AFAIK, nobody builds only libcxx and depend on previous builds on its dependencies. Maybe I should have another variant of LIBCXX_CXX_ABI.

[zibi]

• Make cxx_shared library common for EBCDIC and ASCII build variations.
• Introducing built-libcxxabi to pick up abi headers from previous build.
• Adding the following macros:
  
  LIBCXX_DLL_NAME LIBCXX_SHARED_OUTPUT_NAME LIBCXX_ADDITIONAL_COMPILE_FLAGS LIBCXX_ADDITIONAL_LIBRARIES

[muiez]

libcxxabi/CMakeLists.txt

231

Typo

[zibi]

Putting rational why I opted out for LIBCXX_CXX_ABI=built-libcxxabi.

libcxx/cmake/Modules/HandleLibCXXABI.cmake

137 ↗

(On Diff #459594)

Why do I need another LIBCXX_CXX_ABI?

The system-libcxxabi does not work for us since imported_library cannot find libc++abi.so. This is because find_library works only with .x extension whereas CMAKE_SHARED_LIBRARY_SUFFIX is set to .so. We could use CMAKE_IMPORT_LIBRARY_SUFFIX instead which on z/OS is .x however, cmake does not set CMAKE_IMPORT_LIBRARY_SUFFIX on other platforms (Linux on Power for example).

Forcing .x extension and modifying LIBCXX_CXX_ABI_LIBRARY_PATH to point to previous build, which we want to reuse for ASCII build, the side deck libc++abi.x can be found however, the link step fails with:

ninja: error: 'libcxx-abi-shared-NOTFOUND', needed by 'lib/libc++_a.so', missing and no known rule to make it

Further by removing target_link_libraries(cxx_shared PUBLIC libcxx-abi-shared) will lead to

The archive library liblibcxx-abi-shared.a cannot be found.
This is because we have -llibcxx-abi-shared on the link step whereas we supposed to have <absolude path>/libc++abi.x in LINK_LIBRARIES.

[zibi]

• Make cxx_shared library common for EBCDIC and ASCII build variations.
• Use LIBCXX_CXX_ABI=system-libcxxabi to pick up abi headers and library from previous build.

[zibi]

• Add the following options to documentation:
• LIBCXX_SHARED_OUTPUT_NAME
• LIBCXX_ADDITIONAL_COMPILE_FLAGS
• LIBCXX_ADDITIONAL_LIBRARIES
• LIBCXXABI_ADDITIONAL_LIBRARIES

[zibi]

Looks like my last commit was skipped. I'm adding it now.

[zibi]

Fix just one typo. @dionne I think I addressed all your comments. Please review it.

[zibi]

@ldionne I think I addressed all your comments.
Please review it.

[zibi]

ping @ldionne

[ldionne]

Only a few nits.

libcxx/CMakeLists.txt

472–473

Let's move this to the per-target functions below. Let's put it in cxx_add_basic_build_flags. It should then become target_compile_options(...) and target_link_libraries(...).

libcxx/cmake/caches/s390x32-ibm-zos.cmake

17

This variable doesn't exist, why is it in the cache? If you've got internal-only changes you're keeping downstream, it looks like this should be part of that.

19

Same here regarding internal-only changes.

libcxxabi/CMakeLists.txt

230–234

We should also have LIBCXXABI_ADDITIONAL_COMPILE_FLAGS.

[zibi]

Add LIBCXXABI_ADDITIONAL_COMPILE_FLAGS and other changes requested by Louis.

[zibi]

Ready for another round. FYI @ldionne

[zibi]

Fix CI build.

[zibi]

Fix CI - Documentation

libcxx/CMakeLists.txt

 # TODO: Projects that depend on libc++ should use LIBCXX_GENERATED_INCLUDE_DIR
 # instead of hard-coding include/c++/v1.
 
 set(LIBCXX_INSTALL_INCLUDE_DIR "${CMAKE_INSTALL_INCLUDEDIR}/c++/v1" CACHE PATH
     "Path where target-agnostic libc++ headers should be installed.")
 set(LIBCXX_INSTALL_RUNTIME_DIR "${CMAKE_INSTALL_BINDIR}" CACHE PATH
     "Path where built libc++ runtime libraries should be installed.")
 
+set(LIBCXX_SHARED_OUTPUT_NAME "c++" CACHE STRING
+    "Output name for the shared libc++ runtime library.")
+
 if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND NOT APPLE)
   set(LIBCXX_LIBRARY_DIR ${LLVM_LIBRARY_OUTPUT_INTDIR}/${LLVM_DEFAULT_TARGET_TRIPLE})
   set(LIBCXX_GENERATED_INCLUDE_DIR "${LLVM_BINARY_DIR}/include/c++/v1")
   set(LIBCXX_GENERATED_INCLUDE_TARGET_DIR "${LLVM_BINARY_DIR}/include/${LLVM_DEFAULT_TARGET_TRIPLE}/c++/v1")
   set(LIBCXX_INSTALL_LIBRARY_DIR lib${LLVM_LIBDIR_SUFFIX}/${LLVM_DEFAULT_TARGET_TRIPLE} CACHE PATH
       "Path where built libc++ libraries should be installed.")
   set(LIBCXX_INSTALL_INCLUDE_TARGET_DIR "${CMAKE_INSTALL_INCLUDEDIR}/${LLVM_DEFAULT_TARGET_TRIPLE}/c++/v1" CACHE PATH
       "Path where target-specific libc++ headers should be installed.")

 # They are intended for use as follows:
 # LIBCXX_CXX_FLAGS: General flags for both the compiler and linker.
 # LIBCXX_COMPILE_FLAGS: Compile only flags.
 # LIBCXX_LINK_FLAGS: Linker only flags.
 # LIBCXX_LIBRARIES: libraries libc++ is linked to.
 set(LIBCXX_COMPILE_FLAGS "")
 set(LIBCXX_LINK_FLAGS "")
 set(LIBCXX_LIBRARIES "")
+set(LIBCXX_ADDITIONAL_COMPILE_FLAGS "" CACHE STRING
+    "Additional Compile only flags which can be provided in cache")
+set(LIBCXX_ADDITIONAL_LIBRARIES "" CACHE STRING
+    "Additional libraries libc++ is linked to which can be provided in cache")
 
 # Include macros for adding and removing libc++ flags.
 include(HandleLibcxxFlags)
 
 # Target flags ================================================================
 # These flags get added to CMAKE_CXX_FLAGS and CMAKE_C_FLAGS so that
 # 'config-ix' use them during feature checks. It also adds them to both
 # 'LIBCXX_COMPILE_FLAGS' and 'LIBCXX_LINK_FLAGS'
-if(ZOS)
-  add_target_flags_if_supported("-fzos-le-char-mode=ebcdic")
-endif()
 
 if (${CMAKE_SYSTEM_NAME} MATCHES "AIX")
   add_target_flags_if_supported("-mdefault-visibility-export-mapping=explicit")
   set(CMAKE_AIX_EXPORT_ALL_SYMBOLS OFF)
 endif()
 
 # Configure compiler.
 include(config-ix)
 
 # Configure coverage options.
 if (LIBCXX_GENERATE_COVERAGE)
   include(CodeCoverage)
   set(CMAKE_BUILD_TYPE "COVERAGE" CACHE STRING "" FORCE)
 endif()
 
function(cxx_add_basic_build_flags target)
   if (C_SUPPORTS_COMMENT_LIB_PRAGMA)
     if (LIBCXX_HAS_PTHREAD_LIB)
       target_compile_definitions(${target} PRIVATE -D_LIBCPP_LINK_PTHREAD_LIB)
     endif()
     if (LIBCXX_HAS_RT_LIB)
       target_compile_definitions(${target} PRIVATE -D_LIBCPP_LINK_RT_LIB)
     endif()
   endif()
+  target_compile_options(${target} PUBLIC "${LIBCXX_ADDITIONAL_COMPILE_FLAGS}")
 endfunction()
 
 # Warning flags ===============================================================
 function(cxx_add_warning_flags target)
   target_compile_definitions(${target} PUBLIC -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
   if (MSVC)
     # -W4 is the cl.exe/clang-cl equivalent of -Wall. (In cl.exe and clang-cl,
     # -Wall is equivalent to -Weverything in GCC style compiler drivers.)
if (LIBCXX_TARGETING_MSVC)
     # Required for standards-complaint wide character formatting functions
     # (e.g. `printfw`/`scanfw`)
     target_link_libraries(${target} PRIVATE iso_stdio_wide_specifiers)
   endif()
 
   if (ANDROID AND ANDROID_PLATFORM_LEVEL LESS 21)
     target_link_libraries(${target} PUBLIC android_support)
   endif()
+  target_link_libraries(${target} PUBLIC "${LIBCXX_ADDITIONAL_LIBRARIES}")
 endfunction()
 
 # Windows-related flags =======================================================
 function(cxx_add_windows_flags target)
   if(WIN32 AND NOT MINGW)
     target_compile_definitions(${target} PRIVATE
                                  # Ignore the -MSC_VER mismatch, as we may build
                                  # with a different compatibility version.

libcxx/cmake/caches/s390x-ibm-zos-ascii.cmake

+# Common
+set(CMAKE_BUILD_TYPE Release CACHE STRING "")
+
+set(LIBCXX_ENABLE_ABI_LINKER_SCRIPT OFF CACHE BOOL "")
+set(LIBCXX_ENABLE_SHARED ON CACHE BOOL "")
+set(LIBCXX_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXX_INCLUDE_TESTS OFF CACHE BOOL "")
+set(LIBCXX_INSTALL_HEADERS OFF CACHE BOOL "")
+set(LIBCXX_INCLUDE_BENCHMARKS OFF CACHE BOOL "")
+
+set(LIBCXXABI_ENABLE_SHARED OFF CACHE BOOL "")
+set(LIBCXXABI_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXXABI_INCLUDE_TESTS OFF CACHE BOOL "")
+
+# Target Specific
+set(LIBCXX_DLL_NAME CRTEQCXS CACHE STRING "")
+set(LIBCXX_SHARED_OUTPUT_NAME "c++_a" CACHE STRING
+    "Output name for the shared libc++ runtime library.")
+set(LIBCXX_CXX_ABI system-libcxxabi CACHE STRING "")
+
+set(LIBCXX_ADDITIONAL_COMPILE_FLAGS "-fzos-le-char-mode=ascii" CACHE STRING "")
+set(LIBCXX_ADDITIONAL_LIBRARIES "-L../s390x-ibm-zos/lib -Wl,../s390x-ibm-zos/lib/libunwind.x" CACHE STRING "")

libcxx/cmake/caches/s390x-ibm-zos.cmake

+set(CMAKE_BUILD_TYPE Release CACHE STRING "")
+
+# Common
+set(LIBCXX_ENABLE_ABI_LINKER_SCRIPT OFF CACHE BOOL "")
+set(LIBCXX_ENABLE_SHARED ON CACHE BOOL "")
+set(LIBCXX_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXX_INCLUDE_TESTS ON CACHE BOOL "")
+
+set(LIBCXXABI_ENABLE_SHARED ON CACHE BOOL "")
+set(LIBCXXABI_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXXABI_INCLUDE_TESTS ON CACHE BOOL "")
+
+# Target Specific
+set(LIBCXX_DLL_NAME CRTEQCXE CACHE STRING "")
+
+set(LIBCXXABI_DLL_NAME CRTEQCXA CACHE STRING "")
+set(LIBCXXABI_ADDITIONAL_LIBRARIES "-Wl,lib/libunwind.x" CACHE STRING "")

libcxx/cmake/caches/s390x32-ibm-zos-ascii.cmake

+# Common
+set(CMAKE_BUILD_TYPE Release CACHE STRING "")
+
+set(LIBCXX_ENABLE_ABI_LINKER_SCRIPT OFF CACHE BOOL "")
+set(LIBCXX_ENABLE_SHARED ON CACHE BOOL "")
+set(LIBCXX_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXX_INCLUDE_TESTS OFF CACHE BOOL "")
+set(LIBCXX_INSTALL_HEADERS OFF CACHE BOOL "")
+set(LIBCXX_INCLUDE_BENCHMARKS OFF CACHE BOOL "")
+
+set(LIBCXXABI_ENABLE_SHARED OFF CACHE BOOL "")
+set(LIBCXXABI_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXXABI_INCLUDE_TESTS OFF CACHE BOOL "")
+
+# Target Specific
+set(LIBCXX_DLL_NAME CRTEHCXS CACHE STRING "")
+set(LIBCXX_SHARED_OUTPUT_NAME "c++_a" CACHE STRING
+    "Output name for the shared libc++ runtime library.")
+set(LIBCXX_CXX_ABI system-libcxxabi CACHE STRING "")
+
+set(LIBCXX_ADDITIONAL_COMPILE_FLAGS "-fzos-le-char-mode=ascii" CACHE STRING "")
+set(LIBCXX_ADDITIONAL_LIBRARIES "-L../s390x32-ibm-zos/lib -Wl,../s390x32-ibm-zos/lib/libunwind.x" CACHE STRING "")

libcxx/cmake/caches/s390x32-ibm-zos.cmake

+# Common
+set(CMAKE_BUILD_TYPE Release CACHE STRING "")
+
+set(LIBCXX_ENABLE_ABI_LINKER_SCRIPT OFF CACHE BOOL "")
+set(LIBCXX_ENABLE_SHARED ON CACHE BOOL "")
+set(LIBCXX_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXX_INCLUDE_TESTS ON CACHE BOOL "")
+
+set(LIBCXXABI_ENABLE_SHARED ON CACHE BOOL "")
+set(LIBCXXABI_ENABLE_STATIC OFF CACHE BOOL "")
+set(LIBCXXABI_INCLUDE_TESTS ON CACHE BOOL "")
+
+# Target Specific
+set(LIBCXX_DLL_NAME CRTEHCXE CACHE STRING "")
+
+set(LIBCXXABI_DLL_NAME CRTEHCXA CACHE STRING "")
+
+set(LIBCXXABI_ADDITIONAL_LIBRARIES "-Wl,lib/libunwind.x" CACHE STRING "")

libcxx/docs/BuildingLibcxx.rst

 .. option:: LIBCXX_INSTALL_INCLUDE_TARGET_DIR:PATH
 
   **Default**: ``include/c++/v1`` or
   ``include/${LLVM_DEFAULT_TARGET_TRIPLE}/c++/v1``
 
   Path where target-specific libc++ headers should be installed. If a relative
   path, relative to ``CMAKE_INSTALL_PREFIX``.
 
+.. option:: LIBCXX_SHARED_OUTPUT_NAME:STRING
+
+  **Default**: ``c++``
+
+  Output name for the shared libc++ runtime library.
+
+.. option:: LIBCXX_ADDITIONAL_COMPILE_FLAGS:STRING
+
+  **Default**: ``""``
+
+  Additional Compile only flags which can be provided in cache.
+
+.. option:: LIBCXX_ADDITIONAL_LIBRARIES:STRING
+
+  **Default**: ``""``
+
+  Additional libraries libc++ is linked to which can be provided in cache.
+
 
 .. _ABI Library Specific Options:
 
 ABI Library Specific Options
 ----------------------------
 
 .. option:: LIBCXX_CXX_ABI:STRING
 

 
 .. option:: LIBCXXABI_USE_LLVM_UNWINDER:BOOL
 
   **Default**: ``OFF``
 
   Build and use the LLVM unwinder. Note: This option can only be used when
   libc++abi is the C++ ABI library used.
 
+.. option:: LIBCXXABI_ADDITIONAL_COMPILE_FLAGS:STRING
+
+  **Default**: ``""``
+
+  Additional Compile only flags which can be provided in cache.
+
+.. option:: LIBCXXABI_ADDITIONAL_LIBRARIES:STRING
+
+  **Default**: ``""``
+
+  Additional libraries libc++abi is linked to which can be provided in cache.
+
 
 libc++ Feature Options
 ----------------------
 
 .. option:: LIBCXX_ENABLE_EXCEPTIONS:BOOL
 
   **Default**: ``ON``
 

libcxx/src/CMakeLists.txt

 split_list(LIBCXX_LINK_FLAGS)
 
 # Build the shared library.
 if (LIBCXX_ENABLE_SHARED)
   add_library(cxx_shared SHARED ${exclude_from_all} ${LIBCXX_SOURCES} ${LIBCXX_HEADERS})
   target_include_directories(cxx_shared PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
   target_link_libraries(cxx_shared PUBLIC cxx-headers
                                    PRIVATE ${LIBCXX_LIBRARIES})
   set_target_properties(cxx_shared
     PROPERTIES
       COMPILE_FLAGS "${LIBCXX_COMPILE_FLAGS}"
       LINK_FLAGS    "${LIBCXX_LINK_FLAGS}"
-      OUTPUT_NAME   "c++"
+      OUTPUT_NAME   "${LIBCXX_SHARED_OUTPUT_NAME}"
       VERSION       "${LIBCXX_LIBRARY_VERSION}"
       SOVERSION     "${LIBCXX_ABI_VERSION}"
       DEFINE_SYMBOL ""
   )
   cxx_add_common_build_flags(cxx_shared)
   cxx_set_common_defines(cxx_shared)
 
+  if(ZOS)
+    add_custom_command(TARGET cxx_shared POST_BUILD
+      COMMAND
+        ${LIBCXX_SOURCE_DIR}/utils/zos_rename_dll_side_deck.sh
+        $<TARGET_LINKER_FILE_NAME:cxx_shared> $<TARGET_FILE_NAME:cxx_shared> "${LIBCXX_DLL_NAME}"
+      COMMENT "Rename dll name inside the side deck file"
+      WORKING_DIRECTORY $<TARGET_FILE_DIR:cxx_shared>
+    )
+  endif()
+
   # Link against libc++abi
   if (LIBCXX_STATICALLY_LINK_ABI_IN_SHARED_LIBRARY)
     target_link_libraries(cxx_shared PRIVATE libcxx-abi-shared-objects)
   else()
     target_link_libraries(cxx_shared PUBLIC libcxx-abi-shared)
   endif()
 
   # Maybe re-export symbols from libc++abi
if(WIN32 AND NOT MINGW AND NOT "${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Windows")
     # Since we most likely do not have a mt.exe replacement, disable the
     # manifest bundling.  This allows a normal cmake invocation to pass which
     # will attempt to use the manifest tool to generate the bundled manifest
     set_target_properties(cxx_shared PROPERTIES
                           APPEND_STRING PROPERTY LINK_FLAGS " /MANIFEST:NO")
   endif()
 endif()
 
 set(CMAKE_STATIC_LIBRARY_PREFIX "lib")
 
 # Build the static library.
 if (LIBCXX_ENABLE_STATIC)
   add_library(cxx_static STATIC ${exclude_from_all} ${LIBCXX_SOURCES} ${LIBCXX_HEADERS})
   target_include_directories(cxx_static PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
   target_link_libraries(cxx_static PUBLIC cxx-headers
                                    PRIVATE ${LIBCXX_LIBRARIES}
                                    PRIVATE libcxx-abi-static)
PROPERTIES
       OUTPUT_NAME   "c++external_threads"
   )
 
   target_link_libraries(cxx_external_threads PRIVATE cxx-headers)
 endif()
 
 if (LIBCXX_INSTALL_SHARED_LIBRARY)
   install(TARGETS cxx_shared
     ARCHIVE DESTINATION ${LIBCXX_INSTALL_LIBRARY_DIR} COMPONENT cxx
     LIBRARY DESTINATION ${LIBCXX_INSTALL_LIBRARY_DIR} COMPONENT cxx
     RUNTIME DESTINATION ${LIBCXX_INSTALL_RUNTIME_DIR} COMPONENT cxx)
 endif()
 
 if (LIBCXX_INSTALL_STATIC_LIBRARY)
   install(TARGETS cxx_static
     ARCHIVE DESTINATION ${LIBCXX_INSTALL_LIBRARY_DIR} COMPONENT cxx
     LIBRARY DESTINATION ${LIBCXX_INSTALL_LIBRARY_DIR} COMPONENT cxx

libcxx/utils/zos_rename_dll_side_deck.sh

+#!/usr/bin/env bash
+#
+# Script to rename DLL name within side deck.
+#
+
+# Stops execution if a command or pipeline has an error.
+set -e
+
+sidedeck=$1
+old_dll_name=$2
+new_dll_name=$3
+
+function error() {
+  printf "ERROR: %s\n" "$*"
+  exit 1
+}
+
+function usage() {
+cat <<EOF
+Usage: $(basename $0) <side deck file> <old dll name> <new dll name>:
+          [-h|--help] Display this help and exit.
+EOF
+}
+
+rename_dll_name_inside_side_deck() {
+
+if [[ -z "$sidedeck" || -z "$old_dll_name" || -z "$new_dll_name" ]]; then
+  usage
+  error "All 3 parameters must be specified."
+fi
+
+[[ -f "$sidedeck" ]] || error "The '$sidedeck' file must exists."
+
+old_len=${#old_dll_name}
+new_len=${#new_dll_name}
+
+if (( $new_len > $old_len )); then
+  error "New DLL name $new_dll_name must have $old_len characters or less."
+fi
+
+if ((padding_len=$old_len-$new_len )); then
+  pad=$(printf "%*s" $padding_len "")
+fi
+
+# Touch the temp. file and set the tag to 1047 first so the redirecting statement
+# will write in 1047 and not 819 encoding.
+touch $sidedeck.tmp; chtag -tc1047 $sidedeck.tmp
+sed "/ IMPORT /s/'$old_dll_name/$pad'$new_dll_name/g" $sidedeck > $sidedeck.tmp
+mv $sidedeck.tmp $sidedeck
+}
+
+function main() {
+  rename_dll_name_inside_side_deck
+}
+
+main "$@"
+

libcxxabi/CMakeLists.txt

 # LIBCXXABI_COMPILE_FLAGS: Compile only flags.
 # LIBCXXABI_LINK_FLAGS: Linker only flags.
 # LIBCXXABI_LIBRARIES: libraries libc++abi is linked to.
 
 set(LIBCXXABI_C_FLAGS "")
 set(LIBCXXABI_CXX_FLAGS "")
 set(LIBCXXABI_COMPILE_FLAGS "")
 set(LIBCXXABI_LINK_FLAGS "")
 set(LIBCXXABI_LIBRARIES "")
+set(LIBCXXABI_ADDITIONAL_COMPILE_FLAGS "" CACHE STRING
+    "Additional Compile only flags which can be provided in cache")
+set(LIBCXXABI_ADDITIONAL_LIBRARIES "" CACHE STRING
+    "Additional libraries libc++abi is linked to which can be provided in cache")
 
 # Include macros for adding and removing libc++abi flags.
 include(HandleLibcxxabiFlags)
 
 #===============================================================================
 # Setup Compiler Flags
 #===============================================================================
 
 # Configure target flags
 if(ZOS)
   add_target_flags_if_supported("-fzos-le-char-mode=ebcdic")
 endif()
 
 if (${CMAKE_SYSTEM_NAME} MATCHES "AIX")
   add_target_flags_if_supported("-mdefault-visibility-export-mapping=explicit")
   set(CMAKE_AIX_EXPORT_ALL_SYMBOLS OFF)
 endif()
+add_compile_flags("${LIBCXXABI_ADDITIONAL_COMPILE_FLAGS}")
+add_library_flags("${LIBCXXABI_ADDITIONAL_LIBRARIES}")
 
 # Configure compiler. Must happen after setting the target flags.
 include(config-ix)
 
 if (CXX_SUPPORTS_NOSTDINCXX_FLAG)
   list(APPEND LIBCXXABI_COMPILE_FLAGS -nostdinc++)
   # cmake 3.14 and above remove system include paths that are explicitly
   # passed on the command line.  We build with -nostdinc++ and explicitly add

libcxxabi/src/CMakeLists.txt

if (LIBCXXABI_ENABLE_SHARED)
   add_library(cxxabi_shared SHARED)
   set_target_properties(cxxabi_shared
     PROPERTIES
       LINK_FLAGS "${LIBCXXABI_LINK_FLAGS}"
       OUTPUT_NAME "c++abi"
       SOVERSION "1"
       VERSION "${LIBCXXABI_LIBRARY_VERSION}"
   )
+
+  if (ZOS)
+    add_custom_command(TARGET cxxabi_shared POST_BUILD
+      COMMAND
+        ${LIBCXXABI_LIBCXX_PATH}/utils/zos_rename_dll_side_deck.sh
+        $<TARGET_LINKER_FILE_NAME:cxxabi_shared> $<TARGET_FILE_NAME:cxxabi_shared> "${LIBCXXABI_DLL_NAME}"
+      COMMENT "Rename dll name inside the side deck file"
+      WORKING_DIRECTORY $<TARGET_FILE_DIR:cxxabi_shared>
+    )
+  endif ()
+
   target_link_libraries(cxxabi_shared
     PUBLIC cxxabi_shared_objects
     PRIVATE ${LIBCXXABI_SHARED_LIBRARIES} ${LIBCXXABI_LIBRARIES})
   if (TARGET pstl::ParallelSTL)
     target_link_libraries(cxxabi_shared PUBLIC pstl::ParallelSTL)
   endif()
 
   list(APPEND LIBCXXABI_BUILD_TARGETS "cxxabi_shared")