[AIX][compiler-rt] Build and install standalone libatomic

[lkail]

Summary

On AIX, we have to ship libatomic.a for compatibility. First, a new clang_rt.atomic is added. Second, use added cmake modules for AIX, we are able to build a compatible libatomic.a for AIX. The second step can't be perfectly implemented with cmake now since AIX's archive approach is kinda unique, i.e., archiving shared libraries into a static archive file.

[jsji]

Thanks @lkail ! In general, I think we can try to make the Modules platform dependent.

compiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake

3 ↗

(On Diff #344001)

Can we make *ALL* the code in this file platform independent? Although there are NO other users in other platform, we can make it reusable in case there are similar requests from other platform.

So, please remove AIX_ prefix here.

47 ↗

(On Diff #344001)

Same, remove _aix_ here, also pass down compile_flag and link flag.

60 ↗

(On Diff #344001)

Why this has to be OFF? Isn't SHARED needs PIC? Also AIX defaults to PIC.

61 ↗

(On Diff #344001)

Can we pass the flag into this functions instead of hardcoding them here.

62 ↗

(On Diff #344001)

Same as above, these flags should be passed down from CMakeLists.txt , not hardcoding them here.

74 ↗

(On Diff #344001)

Remove _aix_, also add options to control building 32 bit only, 64 bit only, or both.

80 ↗

(On Diff #344001)

Can we make this another function like generate_export_list?

99 ↗

(On Diff #344001)

ar flags should be passed down, or can have default, and can be overwritten by args.

compiler-rt/lib/builtins/CMakeLists.txt

753

Set the compile flags, linker flags, ar flags here.

[daltenty]

compiler-rt/lib/builtins/CMakeLists.txt

751

Maybe a more generic CMake option (e.g. COMPILER_RT_BUILD_LIBATOMIC) with an appropriate default for AIX would be better here?

[lkail]

Address comments by following method introduced by @daltenty in https://reviews.llvm.org/D101959. Now compiling shared atomic library is platform independent.

[jsji]

I like the part in compiler-rt/lib/builtins/CMakeLists.txt -- it is essentially the same principle of what we suggested yesterday -- make the code platform independent,
but I don't like the overall fact that we need to invoke compiler-rt/utils/aix-install-libatomic.sh as an additional step, and call cmake again...

I know CMake can NOT handle AIX archives well for now, I believe we can just implement that part using ar and add TODO to fix it once CMake support is ready.

What do you think?

[lkail]

What do you think?

Yes, invoking cmake twice does introduce more complexity.

[lkail]

Try to implement all in cmake.

[lkail]

compiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake

3 ↗

(On Diff #344001)

I assume only AIX's binder need these exported symbols, so I make them AIX specific.

61 ↗

(On Diff #344001)

I leave these flags as default options, we can use -DAIX_LIBATOMIC_LINK_FLAGS to change.

[daltenty]

compiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake

54 ↗

(On Diff #344798)

Why not just store this as an .exp file in the source tree if we are just going to write it out? (That would similar to what's in libcxx I believe)

[jsji]

Looks mostly good to me as long as you address David's comments.

compiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake

54 ↗

(On Diff #344798)

Yes, agree.

compiler-rt/lib/builtins/CMakeLists.txt

765

This should be platform independent? So that people can override libatomic link flags to be different than builtin link flags.

[daltenty]

compiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake

62 ↗

(On Diff #344798)

nit: don't some of these duplicate the driver default link flags for building a shared library?

[lkail]

compiler-rt/cmake/Modules/CompilerRTAIXUtils.cmake

62 ↗

(On Diff #344798)

I've checked generated build.ninja

LANGUAGE_COMPILE_FLAGS = -O3 -DNDEBUG
LINK_FLAGS = -Wl,-brtl   -m64 -Wl,-H512 -Wl,-D0      -Wl,-T512 -Wl,-bhalt:4 -Wl,-bernotok      -Wl,-bnoentry -Wl,-bexport:/home/lkail/llvm/dev/llvm-project/compiler-rt/lib/builtins/atomic.exp      -Wl,-bmodtype:SRE -Wl,-lc
LINK_LIBRARIES = -Wl,-blibpath:/usr/lib:/lib
OBJECT_DIR = lib/builtins/CMakeFiles/clang_rt.atomic-dynamic-powerpc64.dir
POST_BUILD = :
PRE_LINK = :

Looks no duplicated item in flags.

[lkail]

• Introduce COMPILER_RT_LIBATOMIC_LINK_FLAGS
• New file atomic.exp under compiler-rt/lib/builtins
• Minor tweak of dependencies

[jsji]

LGTM. Thanks. Please wait one or two days in case there are other feedback.

compiler-rt/lib/builtins/CMakeLists.txt

760

Maybe put atomic.exp into ppc subdir would be better.

[lkail]

• Put atomic.exp in ppc folder
• Move set(BUILTIN_DEPS builtins) under AIX check

compiler-rt/cmake/caches/AIX.cmake

+set(CMAKE_BUILD_TYPE Release CACHE STRING "")
+set(CMAKE_BUILD_WITH_INSTALL_RPATH ON CACHE BOOL "")
+set(CMAKE_SHARED_LINKER_FLAGS
+  "-Wl,-H512 -Wl,-D0 -Wl,-T512 -Wl,-bhalt:4 -Wl,-bernotok -Wl,-bnoentry \
+   -Wl,-bexport:${AIX_LIBATOMIC_EXPORT_LIST} \
+   -Wl,bmodetype:SRE -Wl,-lc"
+  CACHE STRING "")
+
+set(COMPILER_RT_BUILD_LIBATOMIC ON CACHE BOOL "")

compiler-rt/lib/builtins/CMakeLists.txt

if (CAN_TARGET_${arch})
                               SOURCES ${${arch}_SOURCES}
                               DEFS ${BUILTIN_DEFS}
                               CFLAGS ${BUILTIN_CFLAGS}
                               PARENT_TARGET builtins)
     endif ()
   endforeach ()
 endif ()
 
+if (COMPILER_RT_BUILD_LIBATOMIC)
+  add_compiler_rt_runtime(clang_rt.atomic
+                          SHARED
+                          ARCHS ${BUILTIN_SUPPORTED_ARCH}
+                          SOURCES atomic.c
+                          PARENT_TARGET builtins)
+endif()
+
 add_dependencies(compiler-rt builtins)

compiler-rt/utils/aix-install-libatomic.sh

+#!/usr/bin/env bash
+#===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+#===----------------------------------------------------------------------===##
+
+set -e
+
+PROGNAME="$(basename "${0}")"
+
+function error() { printf "error: %s\n" "$*"; exit 1; }
+
+function usage() {
+cat <<EOF
+Usage:
+${PROGNAME} [options]
+
+[-h|--help]                  Display this help and exit.
+
+--llvm-root <DIR>            Path to the root of the LLVM monorepo. Only the compiler-rt
+                             directory is required.
+
+--build-dir <DIR>            Path to the directory to use for building. This will
+                             contain intermediate build products.
+
+--install-dir <DIR>          Path to the directory to install the library to.
+
+--cache <PATH>               The CMake cache to use to control how the library gets built.
+EOF
+}
+
+while [[ $# -gt 0 ]]; do
+    case ${1} in
+        -h|--help)
+            usage
+            exit 0
+            ;;
+        --llvm-root)
+            llvm_root="${2}"
+            shift; shift
+            ;;
+        --build-dir)
+            build_dir="${2}"
+            shift; shift
+            ;;
+        --install-dir)
+            install_dir="${2}"
+            shift; shift
+            ;;
+        --cache)
+            cache="${2}"
+            shift; shift
+            ;;
+        *)
+            error "Unknown argument '${1}'"
+            ;;
+    esac
+done
+
+for arg in llvm_root build_dir install_dir cache; do
+    if [ -z ${!arg+x} ]; then
+        error "Missing required argument '--${arg//_/-}'"
+    elif [ "${!arg}" == "" ]; then
+        error "Argument to --${arg//_/-} must not be empty"
+    fi
+done
+
+# Allow using relative paths
+function realpath() {
+    if [[ $1 = /* ]]; then echo "$1"; else echo "$(pwd)/${1#./}"; fi
+}
+
+for arg in llvm_root build_dir install_dir cache; do
+    path="$(realpath "${!arg}")"
+    eval "${arg}=\"${path}\""
+done
+
+function step() {
+    separator="$(printf "%0.s-" $(seq 1 ${#1}))"
+    echo
+    echo "${separator}"
+    echo "${1}"
+    echo "${separator}"
+}
+
+step "Generating export list"
+exported_routines=$(cat <<EOF
+  __atomic_compare_exchange
+  __atomic_compare_exchange_1
+  __atomic_compare_exchange_2
+  __atomic_compare_exchange_4
+  __atomic_compare_exchange_8
+  __atomic_exchange
+  __atomic_exchange_1
+  __atomic_exchange_2
+  __atomic_exchange_4
+  __atomic_exchange_8
+  __atomic_fetch_add_1
+  __atomic_fetch_add_2
+  __atomic_fetch_add_4
+  __atomic_fetch_add_8
+  __atomic_fetch_and_1
+  __atomic_fetch_and_2
+  __atomic_fetch_and_4
+  __atomic_fetch_and_8
+  __atomic_fetch_or_1
+  __atomic_fetch_or_2
+  __atomic_fetch_or_4
+  __atomic_fetch_or_8
+  __atomic_fetch_sub_1
+  __atomic_fetch_sub_2
+  __atomic_fetch_sub_4
+  __atomic_fetch_sub_8
+  __atomic_fetch_xor_1
+  __atomic_fetch_xor_2
+  __atomic_fetch_xor_4
+  __atomic_fetch_xor_8
+  __atomic_is_lock_free
+  __atomic_load
+  __atomic_load_1
+  __atomic_load_2
+  __atomic_load_4
+  __atomic_load_8
+  __atomic_store
+  __atomic_store_1
+  __atomic_store_2
+  __atomic_store_4
+  __atomic_store_8
+EOF
+)
+export_list=$(realpath "${build_dir}/export_list.atomic")
+echo "${exported_routines}" > "${export_list}"
+
+cmake_path=$(which cmake)
+step "Building libclang_rt.atomic-<arch>.so"
+(cd ${build_dir} && \
+  ${cmake_path} "${llvm_root}/llvm" \
+  -DAIX_LIBATOMIC_EXPORT_LIST="${export_list}" \
+  -C "${cache}" \
+  -GNinja \
+  -DLLVM_ENABLE_RUNTIMES="compiler-rt" \
+  -DLLVM_INSTALL_PREFIX="${install_dir}"
+) && \
+${cmake_path} --build "${build_dir}" --target install-builtins
+
+step "Archive built libclang_rt.atomic-<arch>.so into single libatomic.a"
+# FIXME: Clang's version is undetermined, use wildcard to match.
+atomic_32=$(find \
+            ${install_dir}/lib/clang/*/lib/aix/ \
+            -name libclang_rt.atomic-powerpc.so)
+if [[ -s "${atomic_32}" ]]
+then
+  cp "${atomic_32}" "${build_dir}/libatomic.so.1"
+  ar -X32 r "$(dirname "${atomic_32}")/libatomic.a" "${build_dir}/libatomic.so.1"
+fi
+
+atomic_64=$(find \
+            ${install_dir}/lib/clang/*/lib/aix/ \
+            -name libclang_rt.atomic-powerpc64.so)
+if [[ -s "${atomic_64}" ]]
+then
+  cp "${atomic_64}" "${build_dir}/libatomic.so.1"
+  ar -X64 r "$(dirname "${atomic_64}")/libatomic.a" "${build_dir}/libatomic.so.1"
+fi
+
+rm -f "${build_dir}/libatomic.so.1"