[libc++][test] Improves locale validation.

[Mordante]

Summary

This makes it possible to query the status of a locale in a test. This
allows to use a locale conditionally instead of disabling an entire test
when a locale is missing.

The change is only applied to tests that were disabled in
https://reviews.llvm.org/rGd819703410c2362cbafb60117dab45b182d8b13d

[ldionne]

I think this makes sense. However, I would avoid adding new compiler flags since we try to keep our command-lines as minimal as possible in the test suite.

Instead, I would suggest creating a general mechanism for generating a test_config.h header (which would be the test-suite equivalent of __config_site) and defining those macros there. This header would be generated at lit configuration time (not at CMake configuration time like the __config_site header). This mechanism would be a bit more general and I think it could even be used for other purposes, for example glibc-old-ru_RU-decimal-point could probably be removed in favor of a macro, and we'd gain test coverage in libcxx/test/std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_ru_RU.pass.cpp.

We don't have a precedent for doing this in our test suite, so we'll have to design a way to do it.

[Mordante]

Try a different approach as discussed during review.

[Mordante]

Note this is still a proof-of-concept version not a clean patch.

libcxx/test/std/containers/container.adaptors/container.adaptors.format/format.functions.tests.h

607

Debug to validate the changes work locally.

libcxx/utils/libcxx/test/dsl.py

524

This is a bit hacky to add the path to the includes. I don't want to pollute the source directory with this file.

[arichardson]

libcxx/test/std/utilities/format/format.range/format.range.formatter/format.functions.tests.h

590

How about generating a header with these macros set to 0/1, then typos could be caught by -Werror=undef?

[Mordante]

libcxx/test/std/utilities/format/format.range/format.range.formatter/format.functions.tests.h

590

Typically we use a different way to define these macros

#  ifndef TEST_HAS_NO_LOCALE_fr_FR_UTF_8

I just didn't do that yet to discuss whether this basic approach is feasible.
Then the macro doesn't need 0/1.

[ldionne]

Thanks for looking into this! I am not a huge fan of using the config object to keep state around, but I understand why you had to do it. I'll look into alternative solutions, please give me a bit of time -- let's talk about this again after Issaquah.

libcxx/test/support/test_macros.h

25

I would call the header test_config_site.h.

26–30

I would include this unconditionally. If our setup is messed up and there's no config_site.h, we want to know about it instead of silently having all these macros be undefined.

libcxx/utils/libcxx/test/dsl.py

508–514

libcxx/test/std/containers/container.adaptors/container.adaptors.format/format.functions.tests.h

void test_floating_point(TestFunction check, ExceptionTest check_exception, auto&& input) {
   check(SV("[-42, 0, 1.2, 42]"), SV("{::.{}}"), input, 2);
   check(SV("[-42.500, 0.000, 1.250, 42.500]"), SV("{::.{}f}"), input, 3);
 
   check_exception("The format-spec precision field doesn't contain a value or arg-id", SV("{::.}"), input);
 
   // *** locale-specific form ***
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input); // does not require locales present
 #ifndef TEST_HAS_NO_LOCALIZATION
-// TODO FMT Enable with locale testing active
-#  if 0
+#  ifdef TEST_HAS_LOCALE_fr_FR_UTF_8
   std::locale::global(std::locale(LOCALE_fr_FR_UTF_8));
   check(SV("[-42,5, 0, 1,25, 42,5]"), SV("{::L}"), input);
+#  else
+#    error XXX
+#  endif
 
+#  ifdef TEST_HAS_LOCALE_en_US_UTF_8
   std::locale::global(std::locale(LOCALE_en_US_UTF_8));
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input);
+#  endif
 
   std::locale::global(std::locale::classic());
-#  endif
 #endif // TEST_HAS_NO_LOCALIZATION
 
   // *** type ***
   for (std::basic_string_view<CharT> fmt : fmt_invalid_nested_types<CharT>("aAeEfFgG"))
     check_exception("The format-spec type has a type not supported for a floating-point argument", fmt, input);
 
   // ***** Both have a format-spec
   check(SV("^^[-42.5, ::::0, :1.25, :42.5]^^^"), SV("{:^^33::>5}"), input);

libcxx/test/std/utilities/format/format.range/format.range.fmtset/format.functions.tests.h

void test_floating_point(TestFunction check, ExceptionTest check_exception, auto&& input) {
   check(SV("{-42, 0, 1.2, 42}"), SV("{::.{}}"), input, 2);
   check(SV("{-42.500, 0.000, 1.250, 42.500}"), SV("{::.{}f}"), input, 3);
 
   check_exception("The format-spec precision field doesn't contain a value or arg-id", SV("{::.}"), input);
 
   // *** locale-specific form ***
   check(SV("{-42.5, 0, 1.25, 42.5}"), SV("{::L}"), input); // does not require locales present
 #ifndef TEST_HAS_NO_LOCALIZATION
-// TODO FMT Enable with locale testing active
-#  if 0
+#  ifdef TEST_HAS_LOCALE_fr_FR_UTF_8
   std::locale::global(std::locale(LOCALE_fr_FR_UTF_8));
   check(SV("{-42,5, 0, 1,25, 42,5}"), SV("{::L}"), input);
+#  endif
 
+#  ifdef TEST_HAS_LOCALE_en_US_UTF_8
   std::locale::global(std::locale(LOCALE_en_US_UTF_8));
   check(SV("{-42.5, 0, 1.25, 42.5}"), SV("{::L}"), input);
+#  endif
 
   std::locale::global(std::locale::classic());
-#  endif
 #endif // TEST_HAS_NO_LOCALIZATION
 
   // *** type ***
   for (std::basic_string_view<CharT> fmt : fmt_invalid_nested_types<CharT>("aAeEfFgG"))
     check_exception("The format-spec type has a type not supported for a floating-point argument", fmt, input);
 
   // ***** Both have a format-spec
   check(SV("^^{-42.5, ::::0, :1.25, :42.5}^^^"), SV("{:^^33::>5}"), input);

libcxx/test/std/utilities/format/format.range/format.range.formatter/format.functions.tests.h

void test_floating_point(TestFunction check, ExceptionTest check_exception, auto&& input) {
   check(SV("[-42, 0, 1.2, 42]"), SV("{::.{}}"), input, 2);
   check(SV("[-42.500, 0.000, 1.250, 42.500]"), SV("{::.{}f}"), input, 3);
 
   check_exception("The format-spec precision field doesn't contain a value or arg-id", SV("{::.}"), input);
 
   // *** locale-specific form ***
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input); // does not require locales present
 #ifndef TEST_HAS_NO_LOCALIZATION
-// TODO FMT Enable with locale testing active
-#  if 0
+#  ifdef TEST_HAS_LOCALE_fr_FR_UTF_8
   std::locale::global(std::locale(LOCALE_fr_FR_UTF_8));
   check(SV("[-42,5, 0, 1,25, 42,5]"), SV("{::L}"), input);
+#  endif
 
+#  ifdef TEST_HAS_LOCALE_en_US_UTF_8
   std::locale::global(std::locale(LOCALE_en_US_UTF_8));
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input);
+#  endif
 
   std::locale::global(std::locale::classic());
-#  endif
 #endif // TEST_HAS_NO_LOCALIZATION
 
   // *** type ***
   for (std::basic_string_view<CharT> fmt : fmt_invalid_nested_types<CharT>("aAeEfFgG"))
     check_exception("The format-spec type has a type not supported for a floating-point argument", fmt, input);
 
   // ***** Both have a format-spec
   check(SV("^^[-42.5, ::::0, :1.25, :42.5]^^^"), SV("{:^^33::>5}"), input);

libcxx/test/support/test_macros.h

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef SUPPORT_TEST_MACROS_HPP
-#define SUPPORT_TEST_MACROS_HPP
+#ifndef SUPPORT_TEST_MACROS_H
+#define SUPPORT_TEST_MACROS_H
 
 // Attempt to get STL specific macros like _LIBCPP_VERSION using the most
 // minimal header possible. If we're testing libc++, we should use `<__config>`.
 // If <__config> isn't available, fall back to <ciso646>.
 #ifdef __has_include
 # if __has_include("<__config>")
 #   include <__config>
 #   define TEST_IMP_INCLUDED_HEADER
 # endif
 #endif
 #ifndef TEST_IMP_INCLUDED_HEADER
 #include <ciso646>
 #endif
 
+#ifdef __has_include
+#  if __has_include("config_site.h")
+#    include "config_site.h"
+#  endif
+#endif
+
 #define TEST_STRINGIZE_IMPL(x) #x
 #define TEST_STRINGIZE(x) TEST_STRINGIZE_IMPL(x)
 
 #define TEST_CONCAT1(X, Y) X##Y
 #define TEST_CONCAT(X, Y) TEST_CONCAT1(X, Y)
 
 #ifdef __has_feature
 #define TEST_HAS_FEATURE(X) __has_feature(X)

 #else
 #define TEST_NO_UNIQUE_ADDRESS
 #endif
 
 #ifdef _LIBCPP_SHORT_WCHAR
 #  define TEST_SHORT_WCHAR
 #endif
 
-#endif // SUPPORT_TEST_MACROS_HPP
+#endif // SUPPORT_TEST_MACROS_H

libcxx/utils/libcxx/test/dsl.py

def applyTo(self, config):
     key = self._key
     sub = self._getSub(config)
     config.substitutions.append((key, sub))
 
   def pretty(self, config, litParams):
     return 'add substitution {} = {}'.format(self._key, self._getSub(config))
 
 
+class AddDefine(ConfigAction):
+  """
+  This action adds the given define to the test suite's config_site.
+
+  Currently this only defines macros without a way to specify the macro's
+  value, since there is no usecase for that feature.
+
+  """
+  def __init__(self, name):
+    self._name = name
+
+  def initializeConfig(self, config):
+    config.defines = []
+    config.config_site_root = os.path.join(config.test_exec_root, 'support')
+    if not os.path.exists(config.config_site_root):
+      os.makedirs(config.config_site_root)
+    AddCompileFlag(f"-I {config.config_site_root}").applyTo(config)
+
+  def applyTo(self, config):
+    name = self._name
+    if not hasattr(config, 'defines'):
+      self.initializeConfig(config)
+    config.defines.append(name)
+
+  def pretty(self, config, litParams):
+    return f"add define {self._name}"
+
+
+
 class Feature(object):
   """
   Represents a Lit available feature that is enabled whenever it is supported.
 
   A feature like this informs the test suite about a capability of the compiler,
   platform, etc. Unlike Parameters, it does not make sense to explicitly
   control whether a Feature is enabled -- it should be enabled whenever it
   is supported.

libcxx/utils/libcxx/test/features.py

locales = {
   'zh_CN.UTF-8':     ['zh_CN.UTF-8', 'zh_CN.utf8', 'Chinese_China.936'],
   'fr_CA.ISO8859-1': ['fr_CA.ISO8859-1', 'French_Canada.1252'],
   'cs_CZ.ISO8859-2': ['cs_CZ.ISO8859-2', 'Czech_Czech Republic.1250']
 }
 for locale, alts in locales.items():
   # Note: Using alts directly in the lambda body here will bind it to the value at the
   # end of the loop. Assigning it to a default argument works around this issue.
   DEFAULT_FEATURES.append(Feature(name='locale.{}'.format(locale),
-                                  when=lambda cfg, alts=alts: hasAnyLocale(cfg, alts)))
+                                  when=lambda cfg, alts=alts: hasAnyLocale(cfg, alts),
+                                  actions=[AddDefine('TEST_HAS_LOCALE_{}'.format(re.sub("\W", "_", locale)))]))
 
 
 # Add features representing the target platform name: darwin, linux, windows, etc...
 DEFAULT_FEATURES += [
   Feature(name='darwin', when=lambda cfg: '__APPLE__' in compilerMacros(cfg)),
   Feature(name='windows', when=lambda cfg: '_WIN32' in compilerMacros(cfg)),
   Feature(name='windows-dll', when=lambda cfg: '_WIN32' in compilerMacros(cfg) and programSucceeds(cfg, """
             #include <iostream>

libcxx/utils/libcxx/test/newconfig.py

 
 
 def _getSubstitution(substitution, config):
   for (orig, replacement) in config.substitutions:
     if orig == substitution:
       return replacement
   raise ValueError('Substitution {} is not in the config.'.format(substitution))
 
+CONFIG_SITE_TEMPLATE = """
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SUPPORT_TEST_CONFIG_SITE_H
+#define SUPPORT_TEST_CONFIG_SITE_H
+
+{defines}
+
+#endif // SUPPORT_TEST_CONFIG_SITE_H
+"""
+
+def _writeConfigSite(config):
+  file = open(os.path.join(config.config_site_root, 'config_site.h'), "w")
+  file.write(CONFIG_SITE_TEMPLATE.format(defines = '\n'.join(
+      [
+          "#define {}".format(d) for d in config.defines
+      ])))
+  file.close()
+
 def configure(parameters, features, config, lit_config):
   note = lambda s: lit_config.note("({}) {}".format(config.name, s))
   config.environment = dict(os.environ)
 
   # Apply the actions supplied by parameters to the configuration first, since
   # parameters are things that we request explicitly and which might influence
   # what features are implicitly made available next.
   for param in parameters:
for feature in features:
     actions = feature.getActions(config)
     for action in actions:
       action.applyTo(config)
       if lit_config.debug:
         note("Applied '{}' as a result of implicitly detected feature '{}'".format(
           action.pretty(config, lit_config.params),
           feature.pretty(config)))
 
+  if hasattr(config, 'defines'):
+    _writeConfigSite(config)
+
   # Print the basic substitutions
   for sub in ('%{cxx}', '%{flags}', '%{compile_flags}', '%{link_flags}', '%{exec}'):
     note("Using {} substitution: '{}'".format(sub, _getSubstitution(sub, config)))
 
   # Print all available features
   note("All available features: {}".format(', '.join(config.available_features)))
+
+  # Print all available defines
+  if hasattr(config, 'defines'):
+    note("All available defines: {}".format(', '.join(config.defines)))