[libc++][test] Improves locale validation.

[Mordante]

Summary

This makes it possible to query the status of a locale in a test. This
allows to use a locale conditionally instead of disabling an entire test
when a locale is missing.

The change is only applied to tests that were disabled in
https://reviews.llvm.org/rGd819703410c2362cbafb60117dab45b182d8b13d

[ldionne]

I think this makes sense. However, I would avoid adding new compiler flags since we try to keep our command-lines as minimal as possible in the test suite.

Instead, I would suggest creating a general mechanism for generating a test_config.h header (which would be the test-suite equivalent of __config_site) and defining those macros there. This header would be generated at lit configuration time (not at CMake configuration time like the __config_site header). This mechanism would be a bit more general and I think it could even be used for other purposes, for example glibc-old-ru_RU-decimal-point could probably be removed in favor of a macro, and we'd gain test coverage in libcxx/test/std/localization/locale.categories/category.monetary/locale.money.put/locale.money.put.members/put_long_double_ru_RU.pass.cpp.

We don't have a precedent for doing this in our test suite, so we'll have to design a way to do it.

[Mordante]

Try a different approach as discussed during review.

[Mordante]

Note this is still a proof-of-concept version not a clean patch.

libcxx/test/std/containers/container.adaptors/container.adaptors.format/format.functions.tests.h

607

Debug to validate the changes work locally.

libcxx/utils/libcxx/test/dsl.py

524 ↗

(On Diff #492148)

This is a bit hacky to add the path to the includes. I don't want to pollute the source directory with this file.

[arichardson]

libcxx/test/std/utilities/format/format.range/format.range.formatter/format.functions.tests.h

590

How about generating a header with these macros set to 0/1, then typos could be caught by -Werror=undef?

[Mordante]

libcxx/test/std/utilities/format/format.range/format.range.formatter/format.functions.tests.h

590

Typically we use a different way to define these macros

#  ifndef TEST_HAS_NO_LOCALE_fr_FR_UTF_8

I just didn't do that yet to discuss whether this basic approach is feasible.
Then the macro doesn't need 0/1.

[ldionne]

Thanks for looking into this! I am not a huge fan of using the config object to keep state around, but I understand why you had to do it. I'll look into alternative solutions, please give me a bit of time -- let's talk about this again after Issaquah.

libcxx/test/support/test_macros.h

25 ↗

(On Diff #492148)

I would call the header test_config_site.h.

26–30 ↗

(On Diff #492148)

I would include this unconditionally. If our setup is messed up and there's no config_site.h, we want to know about it instead of silently having all these macros be undefined.

libcxx/utils/libcxx/test/dsl.py

508–514 ↗

(On Diff #492148)

libcxx/test/std/containers/container.adaptors/container.adaptors.format/format.functions.tests.h

void test_floating_point(TestFunction check, ExceptionTest check_exception, auto&& input) {
   check(SV("[-42, 0, 1.2, 42]"), SV("{::.{}}"), input, 2);
   check(SV("[-42.500, 0.000, 1.250, 42.500]"), SV("{::.{}f}"), input, 3);
 
   check_exception("The format-spec precision field doesn't contain a value or arg-id", SV("{::.}"), input);
 
   // *** locale-specific form ***
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input); // does not require locales present
 #ifndef TEST_HAS_NO_LOCALIZATION
-// TODO FMT Enable with locale testing active
-#  if 0
+#  ifdef TEST_HAS_LOCALE_fr_FR_UTF_8
   std::locale::global(std::locale(LOCALE_fr_FR_UTF_8));
   check(SV("[-42,5, 0, 1,25, 42,5]"), SV("{::L}"), input);
+#  endif
 
+#  ifdef TEST_HAS_LOCALE_en_US_UTF_8
   std::locale::global(std::locale(LOCALE_en_US_UTF_8));
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input);
+#  endif
 
   std::locale::global(std::locale::classic());
-#  endif
 #endif // TEST_HAS_NO_LOCALIZATION
 
   // *** type ***
   for (std::basic_string_view<CharT> fmt : fmt_invalid_nested_types<CharT>("aAeEfFgG"))
     check_exception("The format-spec type has a type not supported for a floating-point argument", fmt, input);
 
   // ***** Both have a format-spec
   check(SV("^^[-42.5, ::::0, :1.25, :42.5]^^^"), SV("{:^^33::>5}"), input);

libcxx/test/std/utilities/format/format.range/format.range.fmtset/format.functions.tests.h

void test_floating_point(TestFunction check, ExceptionTest check_exception, auto&& input) {
   check(SV("{-42, 0, 1.2, 42}"), SV("{::.{}}"), input, 2);
   check(SV("{-42.500, 0.000, 1.250, 42.500}"), SV("{::.{}f}"), input, 3);
 
   check_exception("The format-spec precision field doesn't contain a value or arg-id", SV("{::.}"), input);
 
   // *** locale-specific form ***
   check(SV("{-42.5, 0, 1.25, 42.5}"), SV("{::L}"), input); // does not require locales present
 #ifndef TEST_HAS_NO_LOCALIZATION
-// TODO FMT Enable with locale testing active
-#  if 0
+#  ifdef TEST_HAS_LOCALE_fr_FR_UTF_8
   std::locale::global(std::locale(LOCALE_fr_FR_UTF_8));
   check(SV("{-42,5, 0, 1,25, 42,5}"), SV("{::L}"), input);
+#  endif
 
+#  ifdef TEST_HAS_LOCALE_en_US_UTF_8
   std::locale::global(std::locale(LOCALE_en_US_UTF_8));
   check(SV("{-42.5, 0, 1.25, 42.5}"), SV("{::L}"), input);
+#  endif
 
   std::locale::global(std::locale::classic());
-#  endif
 #endif // TEST_HAS_NO_LOCALIZATION
 
   // *** type ***
   for (std::basic_string_view<CharT> fmt : fmt_invalid_nested_types<CharT>("aAeEfFgG"))
     check_exception("The format-spec type has a type not supported for a floating-point argument", fmt, input);
 
   // ***** Both have a format-spec
   check(SV("^^{-42.5, ::::0, :1.25, :42.5}^^^"), SV("{:^^33::>5}"), input);

libcxx/test/std/utilities/format/format.range/format.range.formatter/format.functions.tests.h

void test_floating_point(TestFunction check, ExceptionTest check_exception, auto&& input) {
   check(SV("[-42, 0, 1.2, 42]"), SV("{::.{}}"), input, 2);
   check(SV("[-42.500, 0.000, 1.250, 42.500]"), SV("{::.{}f}"), input, 3);
 
   check_exception("The format-spec precision field doesn't contain a value or arg-id", SV("{::.}"), input);
 
   // *** locale-specific form ***
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input); // does not require locales present
 #ifndef TEST_HAS_NO_LOCALIZATION
-// TODO FMT Enable with locale testing active
-#  if 0
+#  ifdef TEST_HAS_LOCALE_fr_FR_UTF_8
   std::locale::global(std::locale(LOCALE_fr_FR_UTF_8));
   check(SV("[-42,5, 0, 1,25, 42,5]"), SV("{::L}"), input);
+#  endif
 
+#  ifdef TEST_HAS_LOCALE_en_US_UTF_8
   std::locale::global(std::locale(LOCALE_en_US_UTF_8));
   check(SV("[-42.5, 0, 1.25, 42.5]"), SV("{::L}"), input);
+#  endif
 
   std::locale::global(std::locale::classic());
-#  endif
 #endif // TEST_HAS_NO_LOCALIZATION
 
   // *** type ***
   for (std::basic_string_view<CharT> fmt : fmt_invalid_nested_types<CharT>("aAeEfFgG"))
     check_exception("The format-spec type has a type not supported for a floating-point argument", fmt, input);
 
   // ***** Both have a format-spec
   check(SV("^^[-42.5, ::::0, :1.25, :42.5]^^^"), SV("{:^^33::>5}"), input);

libcxx/test/support/platform_support.h

 //   - floating point number string output
 
 #ifndef PLATFORM_SUPPORT_H
 #define PLATFORM_SUPPORT_H
 
 #include "test_macros.h"
 
 // locale names
+//
+// An alternative would be to only define these macros when the test suceeds,
+// that would result in compilation error instead of runtime errors
+//
 #define LOCALE_en_US_UTF_8     "en_US.UTF-8"
 #define LOCALE_fr_FR_UTF_8     "fr_FR.UTF-8"
 #ifdef __linux__
 #    define LOCALE_fr_CA_ISO8859_1 "fr_CA.ISO-8859-1"
 #    define LOCALE_cs_CZ_ISO8859_2 "cs_CZ.ISO-8859-2"
 #elif defined(_WIN32)
 #    define LOCALE_fr_CA_ISO8859_1 "fr-CA"
 #    define LOCALE_cs_CZ_ISO8859_2 "cs-CZ"

libcxx/utils/libcxx/test/features.py

locales = {
   'zh_CN.UTF-8':     ['zh_CN.UTF-8', 'zh_CN.utf8', 'Chinese_China.936'],
   'fr_CA.ISO8859-1': ['fr_CA.ISO8859-1', 'French_Canada.1252'],
   'cs_CZ.ISO8859-2': ['cs_CZ.ISO8859-2', 'Czech_Czech Republic.1250']
 }
 for locale, alts in locales.items():
   # Note: Using alts directly in the lambda body here will bind it to the value at the
   # end of the loop. Assigning it to a default argument works around this issue.
   DEFAULT_FEATURES.append(Feature(name='locale.{}'.format(locale),
-                                  when=lambda cfg, alts=alts: hasAnyLocale(cfg, alts)))
+                                  when=lambda cfg, alts=alts: hasAnyLocale(cfg, alts),
+                                  actions=[AddCompileFlag('-DTEST_HAS_LOCALE_{}'.format(re.sub("\W", "_", locale)))]))
 
 
 # Add features representing the target platform name: darwin, linux, windows, etc...
 DEFAULT_FEATURES += [
   Feature(name='darwin', when=lambda cfg: '__APPLE__' in compilerMacros(cfg)),
   Feature(name='windows', when=lambda cfg: '_WIN32' in compilerMacros(cfg)),
   Feature(name='windows-dll', when=lambda cfg: '_WIN32' in compilerMacros(cfg) and programSucceeds(cfg, """
             #include <iostream>