[runtimes] Add tests for vendor-specific properties

[ldionne]

Summary

Vendors take libc++ and ship it in various ways. Some vendors might
ship it differently from what upstream LLVM does, i.e. the install
location might be different, some ABI properties might differ, etc.

In the past few years, I've come across several instances where
having a place to test some of these properties would have been
incredibly useful. I also just got bitten by the lack of tests
of that kind, so I'm adding some now.

The tests added by this commit for Apple platforms have numerous
TODOs that capture discrepancies between the upstream LLVM CMake
and the slightly-modified build we perform internally to produce
Apple's system libc++. In the future, the goal would be to upstream
all those differences so that it's possible to build a faithful
Apple system libc++ with the upstream LLVM sources only.

But this isn't only useful for Apple - this lays out the path for
any vendor being able to add their own checks (either upstream or
downstream) to libc++.

This is a re-application of 9892d1644f, which was reverted in 138dc27186be
because it broke the build. The issue was that we didn't apply the required
changes to libunwind and our CI didn't notice it because we were not
running the libunwind tests. This has been fixed now, and we're running
the libunwind tests in CI now too.

[leonardchan]

Hi. I think this patch may be what's causing the errors we're seeing on our builders (https://luci-milo.appspot.com/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8834719104333485313):

llvm-lit: /b/s/w/ir/x/w/llvm-project/llvm/utils/lit/lit/TestingConfig.py:102: fatal: unable to parse config file '/b/s/w/ir/x/w/staging/llvm_build/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/libunwind/test/lit.site.cfg', traceback: Traceback (most recent call last):
  File "/b/s/w/ir/x/w/llvm-project/llvm/utils/lit/lit/TestingConfig.py", line 91, in load_from_path
    exec(compile(data, path, 'exec'), cfg_globals, None)
  File "/b/s/w/ir/x/w/staging/llvm_build/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/libunwind/test/lit.site.cfg", line 60, in <module>
    configuration.configure()
  File "/b/s/w/ir/x/w/llvm-project/libcxx/utils/libcxx/test/config.py", line 135, in configure
    self.configure_substitutions()
  File "/b/s/w/ir/x/w/llvm-project/libcxx/utils/libcxx/test/config.py", line 470, in configure_substitutions
    sub.append(('%{install}',       self.quote(self.config.install_root)))
AttributeError: 'TestingConfig' object has no attribute 'install_root'

Would you be able to send out a fix or revert? Thanks.

[uabelho]

We're seeing the same problem as reported above on current trunk, 455b60ccfbf.

Reverting 9892d1644f locally seems to work without conflicts and at least makes my build pass lit tests without problems.

[phosek]

@ldionne This broke all of our builders. Can you look into this issue or revert the change if it cannot be addressed quickly?

[haowei]

Changed reverted in 138dc27186bed4565584387fc49d88b235a1e2ed as author did not respond.
This change broke all of our builders for more than 12hrs.

[ldionne]

In D110736#3034269, @haowei wrote:

Changed reverted in 138dc27186bed4565584387fc49d88b235a1e2ed as author did not respond.
This change broke all of our builders for more than 12hrs.

Thanks for reverting, and sorry for not noticing earlier, but for my defence the first ping happened way after I was off work yesterday. Also, in the future, folks can try to reach out on the #libcxx Discord channel, it's usually the fastest way to get a hold of me (there's a lot of Phabricator emails and this got lost in the noise all morning).

Our CI was green (https://buildkite.com/llvm-project/libcxx-ci/builds/5628). I'm going to look into what happened, I think we are not properly testing libunwind anywhere and that's why the issue went unnoticed.

[ldionne]

Patch to enable libunwind in our CI testing: D110872
Patch to split up part of this patch so it's smaller to land: D110870

[ldionne]

Rebase onto main and fix issues with libunwind

libcxx/cmake/caches/Apple.cmake

 set(LIBCXX_ENABLE_VENDOR_AVAILABILITY_ANNOTATIONS ON CACHE BOOL "")
 set(LIBCXX_ENABLE_INCOMPLETE_FEATURES OFF CACHE BOOL "")
 
 set(LIBCXX_HERMETIC_STATIC_LIBRARY ON CACHE BOOL "")
 set(LIBCXXABI_HERMETIC_STATIC_LIBRARY ON CACHE BOOL "")
 
 set(LIBCXXABI_ENABLE_ASSERTIONS OFF CACHE BOOL "")
 set(LIBCXXABI_ENABLE_FORGIVING_DYNAMIC_CAST ON CACHE BOOL "")
+
+set(LIBCXX_TEST_PARAMS "stdlib=apple-libc++" CACHE STRING "")
+set(LIBCXXABI_TEST_PARAMS "${LIBCXX_TEST_PARAMS}" CACHE STRING "")

libcxx/test/configs/legacy.cfg.in

 @AUTO_GEN_COMMENT@
 
 @SERIALIZED_LIT_PARAMS@
 
 import os
 import site
 
 config.cxx_headers              = "@LIBCXX_GENERATED_INCLUDE_DIR@"
 config.cxx_under_test           = "@CMAKE_CXX_COMPILER@"
 config.project_obj_root         = "@CMAKE_BINARY_DIR@"
+config.install_root             = "@CMAKE_BINARY_DIR@"
 config.libcxx_src_root          = "@LIBCXX_SOURCE_DIR@"
 config.libcxx_obj_root          = "@LIBCXX_BINARY_DIR@"
 config.cxx_library_root         = "@LIBCXX_LIBRARY_DIR@"
 config.abi_library_root         = "@LIBCXX_CXX_ABI_LIBRARY_PATH@"
 config.enable_shared            = @LIBCXX_LINK_TESTS_WITH_SHARED_LIBCXX@
 config.cxx_abi                  = "@LIBCXX_CXX_ABI_LIBNAME@"
 config.configuration_variant    = "@LIBCXX_LIT_VARIANT@"
 config.host_triple              = "@LLVM_HOST_TRIPLE@"

libcxx/test/libcxx/vendor/apple/system-install-properties.sh.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// REQUIRES: stdlib=apple-libc++
+
+// This file checks various properties of the installation of libc++ when built as
+// a system library on Apple platforms.
+//
+// TODO: We should install to `<prefix>/usr` in CMake and check that path instead.
+
+// Make sure we install the libc++ headers in the right location.
+//
+// RUN: stat "%{install}/include/c++/v1/__config"
+
+// Make sure we install libc++.1.dylib in the right location.
+//
+// RUN: stat "%{install}/lib/libc++.1.dylib"
+
+// Make sure we install a symlink from libc++.dylib to libc++.1.dylib.
+//
+// RUN: stat "%{install}/lib/libc++.dylib"
+// RUN: readlink "%{install}/lib/libc++.dylib" | grep "libc++.1.dylib"
+
+// Make sure the install_name is /usr/lib.
+//
+// In particular, make sure we don't use any @rpath in the load commands. When building as
+// a system library, it is important to hardcode the installation paths in the dylib, because
+// various tools like dyld and ld64 will treat us specially if they recognize us as being a
+// system library.
+//
+// TODO: We currently don't do that correctly in the CMake build.
+//
+// XRUNX: otool -L "%{install}/lib/libc++.1.dylib" | grep '/usr/lib/libc++.1.dylib'
+// XRUNX: ! otool -l "%{install}/lib/libc++.1.dylib" | grep -E "LC_RPATH|@loader_path|@rpath"
+
+// Make sure the compatibility_version of libc++ is 1.0.0.
+// Failure to respect this can result in applications not being able to find libc++
+// when they are loaded by dyld, if the compatibility version was bumped.
+//
+// RUN: otool -L "%{install}/lib/libc++.1.dylib" | grep "libc++.1.dylib" | grep "compatibility version 1.0.0"

libcxx/utils/libcxx/test/config.py

class Configuration(object):
     def configure_substitutions(self):
         sub = self.config.substitutions
         sub.append(('%{cxx}', self.quote(self.cxx.path)))
         flags = self.cxx.flags + (self.cxx.modules_flags if self.cxx.use_modules else [])
         compile_flags = self.cxx.compile_flags + (self.cxx.warning_flags if self.cxx.use_warnings else [])
         sub.append(('%{flags}',         ' '.join(map(self.quote, flags))))
         sub.append(('%{compile_flags}', ' '.join(map(self.quote, compile_flags))))
         sub.append(('%{link_flags}',    ' '.join(map(self.quote, self.cxx.link_flags))))
+        sub.append(('%{install}',       self.quote(self.config.install_root)))
 
         codesign_ident = self.get_lit_conf('llvm_codesign_identity', '')
         env_vars = ' '.join('%s=%s' % (k, self.quote(v)) for (k, v) in self.exec_env.items())
         exec_args = [
             '--execdir %T',
             '--codesign_identity "{}"'.format(codesign_ident),
             '--env {}'.format(env_vars)
         ]
         sub.append(('%{exec}', '{} {} -- '.format(self.executor, ' '.join(exec_args))))
 
     def configure_env(self):
         self.config.environment = dict(os.environ)
 
     def add_path(self, dest_env, new_path):
         self.target_info.add_path(dest_env, new_path)

libcxx/utils/libcxx/test/params.py

DEFAULT_PARAMETERS = [
 
   Parameter(name='enable_rtti', choices=[True, False], type=bool, default=True,
             help="Whether to enable RTTI when compiling the test suite.",
             actions=lambda rtti: [] if rtti else [
               AddFeature('no-rtti'),
               AddCompileFlag('-fno-rtti')
             ]),
 
-  Parameter(name='stdlib', choices=['llvm-libc++', 'libstdc++', 'msvc'], type=str, default='llvm-libc++',
+  Parameter(name='stdlib', choices=['llvm-libc++', 'apple-libc++', 'libstdc++', 'msvc'], type=str, default='llvm-libc++',
             help="""The C++ Standard Library implementation being tested.
 
                  Note that this parameter can also be used to encode different 'flavors' of the same
                  standard library, such as libc++ as shipped by a different vendor, if it has different
                  properties worth testing.
 
                  The Standard libraries currently supported are:
                  - llvm-libc++: The 'upstream' libc++ as shipped with LLVM.
+                 - apple-libc++: libc++ as shipped by Apple. This is basically like the LLVM one, but
+                                 there are a few differences like installation paths and the use of
+                                 universal dylibs.
                  - libstdc++: The GNU C++ library typically shipped with GCC.
                  - msvc: The Microsoft implementation of the C++ Standard Library.
                 """,
             actions=lambda stdlib: filter(None, [
               AddFeature('stdlib={}'.format(stdlib)),
               # Also add an umbrella feature 'stdlib=libc++' for all flavors of libc++, to simplify
               # the test suite.
               AddFeature('stdlib=libc++') if re.match('.+-libc\+\+', stdlib) else None

libcxxabi/test/lit.site.cfg.in

 @AUTO_GEN_COMMENT@
 
 @SERIALIZED_LIT_PARAMS@
 
 import os
 import site
 
 config.cxx_under_test           = "@CMAKE_CXX_COMPILER@"
 config.project_obj_root         = "@CMAKE_BINARY_DIR@"
+config.install_root             = "@CMAKE_BINARY_DIR@"
 config.libcxxabi_hdr_root       = "@LIBCXXABI_HEADER_DIR@"
 config.libcxxabi_src_root       = "@LIBCXXABI_SOURCE_DIR@"
 config.libcxxabi_obj_root       = "@LIBCXXABI_BINARY_DIR@"
 config.abi_library_root         = "@LIBCXXABI_LIBRARY_DIR@"
 config.libcxx_src_root          = "@LIBCXXABI_LIBCXX_PATH@"
 config.cxx_headers              = "@LIBCXXABI_LIBCXX_INCLUDES@"
 config.libunwind_headers        = "@LIBCXXABI_LIBUNWIND_INCLUDES_INTERNAL@"
 config.cxx_library_root         = "@LIBCXXABI_LIBCXX_LIBRARY_PATH@"

libcxxabi/test/vendor/apple/system-install-properties.sh.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// REQUIRES: stdlib=apple-libc++
+
+// This file checks various properties of the installation of libc++abi when built
+// as a system library on Apple platforms.
+//
+// TODO: We should install to `<prefix>/usr` in CMake and check that path instead.
+
+// Make sure we install the libc++abi headers in the right location.
+// TODO: We don't currently install them, but we should.
+//
+// XRUNX: stat "%{install}/include/cxxabi.h"
+
+// Make sure we install libc++abi.dylib in the right location.
+//
+// RUN: stat "%{install}/lib/libc++abi.dylib"
+
+// Make sure we don't install a symlink from libc++abi.dylib to libc++abi.1.dylib,
+// unlike what we do for libc++.dylib.
+// TODO: We currently don't do that correctly in the CMake build.
+//
+// XRUNX: ! readlink "%{install}/lib/libc++abi.dylib"
+// XRUNX: ! stat "%{install}/lib/libc++abi.1.dylib"
+
+// Make sure the install_name is /usr/lib.
+//
+// In particular, make sure we don't use any @rpath in the load commands. When building as
+// a system library, it is important to hardcode the installation paths in the dylib, because
+// various tools like dyld and ld64 will treat us specially if they recognize us as being a
+// system library.
+//
+// TODO: We currently don't do that correctly in the CMake build.
+//
+// XRUNX: otool -L "%{install}/lib/libc++abi.dylib" | grep '/usr/lib/libc++abi.dylib'
+// XRUNX: ! otool -l "%{install}/lib/libc++abi.dylib" | grep -E "LC_RPATH|@loader_path|@rpath"
+
+// Make sure the compatibility_version of libc++abi is 1.0.0. Failure to respect this can result
+// in applications not being able to find libc++abi when they are loaded by dyld, if the
+// compatibility version was bumped.
+//
+// RUN: otool -L "%{install}/lib/libc++abi.dylib" | grep "libc++abi.1.dylib" | grep "compatibility version 1.0.0"

libunwind/test/lit.site.cfg.in

 @AUTO_GEN_COMMENT@
 
 @SERIALIZED_LIT_PARAMS@
 
 import os
 import site
 
 config.cxx_under_test           = "@CMAKE_CXX_COMPILER@"
 config.project_obj_root         = "@CMAKE_BINARY_DIR@"
+config.install_root             = "@CMAKE_BINARY_DIR@"
 config.libunwind_src_root       = "@LIBUNWIND_SOURCE_DIR@"
 config.libunwind_obj_root       = "@LIBUNWIND_BINARY_DIR@"
 config.abi_library_root         = "@LIBUNWIND_LIBRARY_DIR@"
 config.libcxx_src_root          = "@LIBUNWIND_LIBCXX_PATH@"
 config.libunwind_headers        = "@LIBUNWIND_SOURCE_DIR@/include"
 config.cxx_library_root         = "@LIBUNWIND_LIBCXX_LIBRARY_PATH@"
 config.llvm_unwinder            = True
 config.builtins_library         = "@LIBUNWIND_BUILTINS_LIBRARY@"