[libcxx] Add deployment knobs to tests (for Apple platforms)

[mehdi_amini]

Summary

The tests for libc++ specify -target on the command-line to the
compiler, but this is problematic for a few reasons.

Firstly, the -target option isn't supported on Apple platforms. Parts
of the triple get dropped and ignored. Instead, software should be
compiled with a combination of the -arch and -m<name>-version-min
options.

Secondly, the generic "darwin" target references a kernel version
instead of a platform version. Each platform has its own independent
versions (with different versions of libc++.1.dylib), independent of the
version of the Darwin kernel.

This commit adds support to the LIT infrastructure for testing against
Apple platforms using -arch and -platform options.

• If the host is not on OS X, or the compiler type is not clang or apple-clang, then this commit has NFC.

• If the host is on OS X and --param=target_triple=... is specified, then a warning is emitted to use arch and platform instead. Besides the warning, there's NFC.

• If the host is on OS X and *no* target-triple is specified, then use the new deployment target logic. This uses two new lit parameters, --param=arch=<arch> and --param=platform=<platform>. <platform> has the form <name>[<version>].
  • By default, arch is auto-detected from clang -dumpmachine, and platform is "macosx".
  • If the platform doesn't have a version:
    • For "macosx", the version is auto-detected from the host system using sw_vers. This may give a different version than the SDK, since new SDKs can be installed on older hosts.
    • Otherwise, the version is auto-detected from the SDK version using xcrun --show-sdk-path.
  • -arch <arch> -m<name>-version-min=<version> is added to the compiler flags.
  • The target triple is computed as <arch>-apple-<platform>. It is *not* passed to clang, but it is available for XFAIL and UNSUPPORTED (as is with_system_cxx_lib=<target>).
  • For convenience, apple-darwin and <arch>-apple-darwin are added to the set of available features.

There were a number of tests marked to XFAIL on x86_64-apple-darwin11
and x86_64-apple-darwin12. I updated these to
x86_64-apple-macosx10.7 and x86_64-apple-macosx10.8.

[dexonsmith]

Does anyone have a problem with this direction? I have commits to
follow that get tests green when run against various releases of OSX
but I need this in place first.

[mclow.lists]

This direction looks fine to me. All the test changes look fine to me.

[EricWF]

LGTM.

test/libcxx/test/config.py

66

Was the omission of self.use_deployment here intentional? Python linters typically get upset by this.

[Eugene.Zelenko]

Looks like patch was not committed.

[Eugene.Zelenko]

It's fine. I just walk through all accepted revisions to find forgotten ones.

[dexonsmith]

Mehdi, are you interested in rebasing this?

[mehdi_amini]

Rebase

[mehdi_amini]

Add back the python changes (files were moved, I lost them in the rebase)

test/libcxx/test/config.py

66

I can see how it would be useful not to add it here: we'd want an exception if it is used without being set since we don't expect it to happen.

[mehdi_amini]

test/libcxx/test/config.py

66

But if it is not considered "pythonic", I'm fine adding it as well :)

[mehdi_amini]

Update: we set deployment by default even when not testing the system library.
It is intentional because a REQUIRE in a test might be because an issue with libc for example.
So for now the default is the OS on which the test runs.

[mehdi_amini]

Was committed a while back in r297798

test/libcxx/test/config.py

def __init__(self, lit_config, config):
         self.config = config
         self.cxx = None
         self.project_obj_root = None
         self.libcxx_src_root = None
         self.libcxx_obj_root = None
         self.cxx_library_root = None
         self.abi_library_root = None
         self.env = {}
         self.use_target = False
         self.use_system_cxx_lib = False
         self.use_clang_verify = False
         self.long_tests = None
         self.execute_external = False
 
     def get_lit_conf(self, name, default=None):
         val = self.lit_config.params.get(name, None)
         if val is None:
def get_lit_bool(self, name, default=None):
         self.lit_config.fatal(
             "parameter '{}' should be true or false".format(name))
 
     def configure(self):
         self.configure_executor()
         self.configure_target_info()
         self.configure_cxx()
         self.configure_triple()
+        self.configure_deployment()
         self.configure_src_root()
         self.configure_obj_root()
         self.configure_cxx_library_root()
         self.configure_use_system_cxx_lib()
         self.configure_use_clang_verify()
         self.configure_execute_external()
         self.configure_ccache()
         self.configure_compile_flags()
def configure_features(self):
         # libc++ as were shipped with a particular triple.
         if self.use_system_cxx_lib:
             self.config.available_features.add(
                 'with_system_cxx_lib=%s' % self.config.target_triple)
 
         # Insert the platform name into the available features as a lower case.
         self.config.available_features.add(target_platform)
 
+        # If we're using deployment, add sub-components of the triple using
+        # "darwin" instead of the platform name.
+        if self.use_deployment:
+            arch, _, _ = self.config.deployment
+            self.config.available_features.add('apple-darwin')
+            self.config.available_features.add(arch + '-apple-darwin')
+
         # Simulator testing can take a really long time for some of these tests
         # so add a feature check so we can REQUIRES: long_tests in them
         self.long_tests = self.get_lit_bool('long_tests')
         if self.long_tests is None:
             # Default to running long tests.
             self.long_tests = True
             self.lit_config.note(
                 "inferred long_tests as: %r" % self.long_tests)
def configure_default_compile_flags(self):
         sysroot = self.get_lit_conf('sysroot')
         if sysroot:
             self.cxx.flags += ['--sysroot', sysroot]
         gcc_toolchain = self.get_lit_conf('gcc_toolchain')
         if gcc_toolchain:
             self.cxx.flags += ['-gcc-toolchain', gcc_toolchain]
         if self.use_target:
             self.cxx.flags += ['-target', self.config.target_triple]
+        if self.use_deployment:
+            arch, name, version = self.config.deployment
+            self.cxx.flags += ['-arch', arch]
+            self.cxx.flags += ['-m' + name + '-version-min=' + version]
 
     def configure_compile_flags_header_includes(self):
         support_path = os.path.join(self.libcxx_src_root, 'test/support')
         self.cxx.compile_flags += ['-I' + support_path]
         self.cxx.compile_flags += ['-include', os.path.join(support_path, 'nasty_macros.hpp')]
         self.configure_config_site_header()
         libcxx_headers = self.get_lit_conf(
             'libcxx_headers', os.path.join(self.libcxx_src_root, 'include'))
def configure_substitutions(self):
         sub.append(('%exec', exec_str))
         # Configure run shortcut
         sub.append(('%run', exec_str + ' %t.exe'))
         # Configure not program substitions
         not_py = os.path.join(self.libcxx_src_root, 'utils', 'not', 'not.py')
         not_str = '%s %s' % (sys.executable, not_py)
         sub.append(('not', not_str))
 
+    def can_use_deployment(self):
+        # Check if the host is on an Apple platform using clang.
+        if not self.target_info.platform() == "darwin":
+            return False
+        if not self.target_info.is_host_macosx():
+            return False
+        if not self.cxx.type.endswith('clang'):
+            return False
+        return True
+
     def configure_triple(self):
         # Get or infer the target triple.
-        self.config.target_triple = self.get_lit_conf('target_triple')
-        self.use_target = bool(self.config.target_triple)
+        target_triple = self.get_lit_conf('target_triple')
+        self.use_target = bool(target_triple)
+
+        # Use deployment if possible.
+        self.use_deployment = not self.use_target and self.can_use_deployment()
+        if self.use_deployment:
+            return
+
+        # Save the triple (and warn on Apple platforms).
+        self.config.target_triple = target_triple
+        if self.use_target and 'apple' in target_triple:
+            self.lit_config.warning(
+                    'consider using arch and platform instead'
+                    ' of target_triple on Apple platforms')
+
         # If no target triple was given, try to infer it from the compiler
         # under test.
         if not self.use_target:
             target_triple = self.cxx.getTriple()
             # Drop sub-major version components from the triple, because the
             # current XFAIL handling expects exact matches for feature checks.
             # Example: x86_64-apple-darwin14.0.0 -> x86_64-apple-darwin14
             # The 5th group handles triples greater than 3 parts
             # (ex x86_64-pc-linux-gnu).
             target_triple = re.sub(r'([^-]+)-([^-]+)-([^.]+)([^-]*)(.*)',
                                    r'\1-\2-\3\5', target_triple)
             # linux-gnu is needed in the triple to properly identify linuxes
             # that use GLIBC. Handle redhat and opensuse triples as special
             # cases and append the missing `-gnu` portion.
             if (target_triple.endswith('redhat-linux') or
                 target_triple.endswith('suse-linux')):
                 target_triple += '-gnu'
             self.config.target_triple = target_triple
             self.lit_config.note(
                 "inferred target_triple as: %r" % self.config.target_triple)
 
+    def configure_deployment(self):
+        assert not self.use_deployment is None
+        assert not self.use_target is None
+        if not self.use_deployment:
+            # Warn about ignored parameters.
+            if self.get_lit_conf('arch'):
+                self.lit_config.warning('ignoring arch, using target_triple')
+            if self.get_lit_conf('platform'):
+                self.lit_config.warning('ignoring platform, using target_triple')
+            return
+
+        assert not self.use_target
+        assert self.target_info.is_host_macosx()
+
+        # Always specify deployment explicitly on Apple platforms, since
+        # otherwise a platform is picked up from the SDK.  If the SDK version
+        # doesn't match the system version, tests that use the system library
+        # may fail spuriously.
+        arch = self.get_lit_conf('arch')
+        if not arch:
+            arch = self.cxx.getTriple().split('-', 1)[0]
+            self.lit_config.note("inferred arch as: %r" % arch)
+
+        inferred_platform, name, version = self.target_info.get_platform()
+        if inferred_platform:
+            self.lit_config.note("inferred platform as: %r" % (name + version))
+        self.config.deployment = (arch, name, version)
+
+        # Set the target triple for use by lit.
+        self.config.target_triple = arch + '-apple-' + name + version
+        self.lit_config.note(
+            "computed target_triple as: %r" % self.config.target_triple)
+
     def configure_env(self):
         self.target_info.configure_env(self.env)

test/libcxx/test/target_info.py

 #===----------------------------------------------------------------------===//
 #
 #                     The LLVM Compiler Infrastructure
 #
 # This file is dual licensed under the MIT and the University of Illinois Open
 # Source Licenses. See LICENSE.TXT for details.
 #
 #===----------------------------------------------------------------------===//
 
 import importlib
 import lit.util  # pylint: disable=import-error,no-name-in-module
 import locale
 import os
 import platform
+import re
 import sys
 
 class DefaultTargetInfo(object):
     def __init__(self, full_config):
         self.full_config = full_config
 
     def platform(self):
         return sys.platform.lower().strip()
for loc in locales:
                                'your platform. Some tests will be '
                                'unsupported.'.format(loc))
 
 
 class DarwinLocalTI(DefaultTargetInfo):
     def __init__(self, full_config):
         super(DarwinLocalTI, self).__init__(full_config)
 
+    def is_host_macosx(self):
+        name = lit.util.capture(['sw_vers', '-productName']).strip()
+        return name == "Mac OS X"
+
+    def get_macosx_version(self):
+        assert self.is_host_macosx()
+        version = lit.util.capture(['sw_vers', '-productVersion']).strip()
+        version = re.sub(r'([0-9]+\.[0-9]+)(\..*)?', r'\1', version)
+        return version
+
+    def get_sdk_version(self, name):
+        assert self.is_host_macosx()
+        cmd = ['xcrun', '--sdk', name, '--show-sdk-path']
+        try:
+            out = lit.util.capture(cmd).strip()
+        except OSError:
+            pass
+
+        if not out:
+            self.full_config.lit_config.fatal(
+                    "cannot infer sdk version with: %r" % cmd)
+
+        return re.sub(r'.*/[^0-9]+([0-9.]+)\.sdk', r'\1', out)
+
+    def get_platform(self):
+        platform = self.full_config.get_lit_conf('platform')
+        if platform:
+            platform = re.sub(r'([^0-9]+)([0-9\.]*)', r'\1-\2', platform)
+            name, version = tuple(platform.split('-', 1))
+        else:
+            name = 'macosx'
+            version = None
+
+        if version:
+            return (False, name, version)
+
+        # Infer the version, either from the SDK or the system itself.  For
+        # macosx, ignore the SDK version; what matters is what's at
+        # /usr/lib/libc++.dylib.
+        if name == 'macosx':
+            version = self.get_macosx_version()
+        else:
+            version = self.get_sdk_version(name)
+        return (True, name, version)
+
     def add_locale_features(self, features):
         add_common_locales(features, self.full_config.lit_config)
 
     def add_cxx_compile_flags(self, flags):
+        if self.full_config.use_deployment:
+            _, name, _ = self.full_config.config.deployment
+            cmd = ['xcrun', '--sdk', name, '--show-sdk-path']
+        else:
+            cmd = ['xcrun', '--show-sdk-path']
         try:
-            out = lit.util.capture(['xcrun', '--show-sdk-path']).strip()
+            out = lit.util.capture(cmd).strip()
             res = 0
         except OSError:
             res = -1
         if res == 0 and out:
             sdk_path = out
             self.full_config.lit_config.note('using SDKROOT: %r' % sdk_path)
             flags += ["-isysroot", sdk_path]
 

test/std/localization/locale.categories/category.ctype/ctype_base.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <locale>
 
 // class ctype_base
 // {
 // public:
 //     typedef T mask;
 //

test/std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_1.pass.cpp

 // REQUIRES: locale.en_US.UTF-8
 
 // <locale>
 
 // template <class charT> class ctype_byname;
 
 // charT tolower(charT) const;
 
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 // XFAIL: linux
 
 #include <locale>
 #include <cassert>
 
 #include "platform_support.h" // locale name macros
 
 int main()

test/std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_many.pass.cpp

 // REQUIRES: locale.en_US.UTF-8
 
 // <locale>
 
 // template <class charT> class ctype_byname;
 
 // const charT* tolower(charT* low, const charT* high) const;
 
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 // XFAIL: linux
 
 #include <locale>
 #include <string>
 #include <cassert>
 
 #include "platform_support.h" // locale name macros
 

test/std/localization/locale.categories/category.ctype/locale.ctype.byname/toupper_1.pass.cpp

 // REQUIRES: locale.en_US.UTF-8
 
 // <locale>
 
 // template <class charT> class ctype_byname;
 
 // charT toupper(charT) const;
 
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 // XFAIL: linux
 
 #include <locale>
 #include <cassert>
 
 #include "platform_support.h" // locale name macros
 
 int main()

test/std/localization/locale.categories/category.ctype/locale.ctype.byname/toupper_many.pass.cpp

 // REQUIRES: locale.en_US.UTF-8
 
 // <locale>
 
 // template <class charT> class ctype_byname;
 
 // const charT* toupper(charT* low, const charT* high) const;
 
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 // XFAIL: linux
 
 #include <locale>
 #include <string>
 #include <cassert>
 
 #include "platform_support.h" // locale name macros
 

test/std/localization/locale.categories/category.monetary/locale.moneypunct/types.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <locale>
 
 // template <class _CharT, bool _International = false>
 // class moneypunct
 //     : public locale::facet,
 //       public money_base
 // {

test/std/localization/locale.categories/category.numeric/locale.num.get/facet.num.get.members/get_double.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <locale>
 
 // class num_get<charT, InputIterator>
 
 // iter_type get(iter_type in, iter_type end, ios_base&,
 //               ios_base::iostate& err, double& v) const;
 

test/std/localization/locales/locale/locale.types/locale.category/category.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <locale>
 
 // typedef int category;
 
 #include <locale>
 #include <type_traits>
 #include <cassert>

test/std/re/re.traits/translate_nocase.pass.cpp

 // <regex>
 
 // template <class charT> struct regex_traits;
 
 // charT translate_nocase(charT c) const;
 
 // REQUIRES: locale.en_US.UTF-8
 
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // TODO: investigation needed
 // XFAIL: linux-gnu
 
 #include <regex>
 #include <cassert>
 
 #include "platform_support.h"

test/std/strings/string.conversions/stof.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // XFAIL: libcpp-no-exceptions
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <string>
 
 // float stof(const string& str, size_t *idx = 0);
 // float stof(const wstring& str, size_t *idx = 0);
 
 #include <string>
 #include <cmath>

test/std/strings/string.conversions/stol.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // XFAIL: libcpp-no-exceptions
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <string>
 
 // long stol(const string& str, size_t *idx = 0, int base = 10);
 // long stol(const wstring& str, size_t *idx = 0, int base = 10);
 
 #include <string>
 #include <cassert>

test/std/strings/string.conversions/stoll.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // XFAIL: libcpp-no-exceptions
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <string>
 
 // long long stoll(const string& str, size_t *idx = 0, int base = 10);
 // long long stoll(const wstring& str, size_t *idx = 0, int base = 10);
 
 #include <string>
 #include <cassert>

test/std/strings/string.conversions/stoul.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // XFAIL: libcpp-no-exceptions
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <string>
 
 // unsigned long stoul(const string& str, size_t *idx = 0, int base = 10);
 // unsigned long stoul(const wstring& str, size_t *idx = 0, int base = 10);
 
 #include <string>
 #include <cassert>

test/std/strings/string.conversions/stoull.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // XFAIL: libcpp-no-exceptions
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <string>
 
 // unsigned long long stoull(const string& str, size_t *idx = 0, int base = 10);
 // unsigned long long stoull(const wstring& str, size_t *idx = 0, int base = 10);
 
 #include <string>
 #include <cassert>

test/std/thread/futures/futures.future_error/what.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 
 // LWG 2056 changed the values of future_errc, so if we're using new headers
 // with an old library we'll get incorrect messages.
 //
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 // XFAIL: with_system_cxx_lib=x86_64-apple-darwin13
 
 // <future>
 
 // class future_error
 
 // const char* what() const throw();
 

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_compare_exchange_strong.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // bool
 // atomic_compare_exchange_strong(shared_ptr<T>* p, shared_ptr<T>* v,

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_compare_exchange_strong_explicit.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // bool
 // atomic_compare_exchange_strong_explicit(shared_ptr<T>* p, shared_ptr<T>* v,

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_compare_exchange_weak.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // bool
 // atomic_compare_exchange_weak(shared_ptr<T>* p, shared_ptr<T>* v,

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_compare_exchange_weak_explicit.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // bool
 // atomic_compare_exchange_weak_explicit(shared_ptr<T>* p, shared_ptr<T>* v,

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_exchange.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // shared_ptr<T>
 // atomic_exchange(shared_ptr<T>* p, shared_ptr<T> r)

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_exchange_explicit.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // shared_ptr<T>
 // atomic_exchange_explicit(shared_ptr<T>* p, shared_ptr<T> r)

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_load.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // shared_ptr<T>
 // atomic_load(const shared_ptr<T>* p)

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_load_explicit.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // shared_ptr<T>
 // atomic_load_explicit(const shared_ptr<T>* p, memory_order mo)

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_store.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // void
 // atomic_store(shared_ptr<T>* p, shared_ptr<T> r)

test/std/utilities/memory/util.smartptr/util.smartptr.shared.atomic/atomic_store_explicit.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <memory>
 
 // shared_ptr
 
 // template <class T>
 // void
 // atomic_store_explicit(shared_ptr<T>* p, shared_ptr<T> r, memory_order mo)

test/std/utilities/time/time.clock/time.clock.hires/consistency.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <chrono>
 
 // high_resolution_clock
 
 // check clock invariants
 
 #include <chrono>

test/std/utilities/time/time.clock/time.clock.steady/consistency.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 // UNSUPPORTED: libcpp-has-no-monotonic-clock
 
 // <chrono>
 
 // steady_clock
 
 // check clock invariants
 

test/std/utilities/time/time.clock/time.clock.system/consistency.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This test uses new symbols that were not defined in the libc++ shipped on
 // darwin11 and darwin12:
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin11
-// XFAIL: with_system_cxx_lib=x86_64-apple-darwin12
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.7
+// XFAIL: with_system_cxx_lib=x86_64-apple-macosx10.8
 
 // <chrono>
 
 // system_clock
 
 // check clock invariants
 
 #include <chrono>