[libc++][test] Use the Japanese locale.

[Mordante]

Summary

This change is done to see whether all platforms have a CI with the
Japanese locale installed.

This wires in the locale in the tests and uses it in one test. This is
a preparation for the tests of the chrono formatters.

[Mordante]

Only put up for review to get an initial view which CI jobs have a Japanese locale installed.

libcxx/test/std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp

15

Process this TODO.

libcxx/utils/ci/buildkite-pipeline.yml

20 ↗

(On Diff #419163)

Process this TODO.

[Mordante]

Add debug info to look at unexpected CI failures.

[mstorsjo]

libcxx/test/std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp

86

FWIW in other locale tests, we've used unicode escapes like \u0123 instead of spelling them out as utf8 in the source.

[Mordante]

The Linux CI changes are committed, but it will take some time before they are propagated to the build nodes.
I will look in a little while whether the propagation has been completed.

libcxx/test/std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp

86

Good point. I'll address that in the next revision.

[Mordante]

Address a review comment.
Rebased to see whether the CI Docker images are updated.

[Mordante]

Try to fix the CI.

[Mordante]

Attempt to fix CI

[Mordante]

Fix a copy paste error.

[Mordante]

Test requires is used.

[Mordante]

Rebased.
The previous runs are as expected, prepare for review by removing unwanted changes.

[Mordante]

This is ready for review, assuming the CI passes ;-)

libcxx/test/std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp

14

For info I tested with this value set and an assert(false); in the tests to verify all platforms support Japanese. Only on AIX it's unsupported. However this platform has a lot (all?) locale tests disabled.

The build can be found here
https://buildkite.com/llvm-project/libcxx-ci/builds/10064

[Mordante]

Rebased to test status.

[Mordante]

Rebase to trigger CI.

[ldionne]

LGTM, but have you verified that we were running that test in the CI? Since it REQUIRES: locale.ja_JP.UTF-8, we would simply skip it if the CI node didn't support that locale. Please make sure that's not what's happening, as we would simply lose coverage of that test.

[Mordante]

In D122738#3439685, @ldionne wrote:

LGTM, but have you verified that we were running that test in the CI? Since it REQUIRES: locale.ja_JP.UTF-8, we would simply skip it if the CI node didn't support that locale. Please make sure that's not what's happening, as we would simply lose coverage of that test.

Thanks for the review!

I indeed have verified that, per https://reviews.llvm.org/D122738#inline-1180541

For info I tested with this value set and an assert(false); in the tests to verify all platforms support Japanese. Only on AIX it's unsupported. However this platform has a lot (all?) locale tests disabled.

The build can be found here
https://buildkite.com/llvm-project/libcxx-ci/builds/10064

libcxx/test/std/localization/locale.categories/category.time/locale.time.put.byname/put1.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // NetBSD does not support LC_TIME at the moment
 // XFAIL: netbsd
 
 // REQUIRES: locale.en_US.UTF-8
 // REQUIRES: locale.fr_FR.UTF-8
+// REQUIRES: locale.ja_JP.UTF-8
 
 // <locale>
 
 // template <class CharT, class OutputIterator = ostreambuf_iterator<CharT> >
 // class time_put_byname
 //     : public time_put<CharT, OutputIterator>
 // {
 // public:
 //     explicit time_put_byname(const char* nm, size_t refs = 0);
std::ios ios(0);
         std::string pat("Today is %A which is abbreviated '%a'.");
         cpp17_output_iterator<char*> iter =
             f.put(cpp17_output_iterator<char*>(str), ios, '*', &t, pat.data(), pat.data() + pat.size());
         std::string ex(str, base(iter));
         assert((ex == "Today is Samedi which is abbreviated 'Sam'.")||
                (ex == "Today is samedi which is abbreviated 'sam'." )||
                (ex == "Today is samedi which is abbreviated 'sam.'."));
     }
-
+    {
+        const my_facet f(LOCALE_ja_JP_UTF_8, 1);
+        std::string pat("Today is %A which is the %uth day or alternatively %Ou.");
+        cpp17_output_iterator<char*> iter =
+            f.put(cpp17_output_iterator<char*>(str), ios, '*', &t, pat.data(), pat.data() + pat.size());
+        std::string ex(str, base(iter));
+#if defined(_WIN32) ||  defined(__APPLE__)
+    // These platforms have no alternative
+        assert(ex == "Today is \xE5\x9C\x9F\xE6\x9B\x9C\xE6\x97\xA5 which is the 6th day or alternatively 6.");
+#else
+        assert(ex == "Today is \xE5\x9C\x9F\xE6\x9B\x9C\xE6\x97\xA5 which is the 6th day or alternatively \xE5\x85\xAD.");
+#endif
+    }
   return 0;
 }

libcxx/test/support/platform_support.h

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // Define a bunch of macros that can be used in the tests instead of

 #    define LOCALE_cs_CZ_ISO8859_2 "cs_CZ.ISO-8859-2"
 #elif defined(_WIN32)
 #    define LOCALE_fr_CA_ISO8859_1 "fr-CA"
 #    define LOCALE_cs_CZ_ISO8859_2 "cs-CZ"
 #else
 #    define LOCALE_fr_CA_ISO8859_1 "fr_CA.ISO8859-1"
 #    define LOCALE_cs_CZ_ISO8859_2 "cs_CZ.ISO8859-2"
 #endif
+#define LOCALE_ja_JP_UTF_8     "ja_JP.UTF-8"
 #define LOCALE_ru_RU_UTF_8     "ru_RU.UTF-8"
 #define LOCALE_zh_CN_UTF_8     "zh_CN.UTF-8"
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <codecvt>
 #include <locale>
 #include <string>

libcxx/utils/libcxx/test/features.py

 
 
 # Mapping from canonical locale names (used in the tests) to possible locale
 # names on various systems. Each locale is considered supported if any of the
 # alternative names is supported.
 locales = {
   'en_US.UTF-8':     ['en_US.UTF-8', 'en_US.utf8', 'English_United States.1252'],
   'fr_FR.UTF-8':     ['fr_FR.UTF-8', 'fr_FR.utf8', 'French_France.1252'],
+  'ja_JP.UTF-8':     ['ja_JP.UTF-8', 'ja_JP.utf8', 'Japanese_Japan.923'],
   'ru_RU.UTF-8':     ['ru_RU.UTF-8', 'ru_RU.utf8', 'Russian_Russia.1251'],
   'zh_CN.UTF-8':     ['zh_CN.UTF-8', 'zh_CN.utf8', 'Chinese_China.936'],
   'fr_CA.ISO8859-1': ['fr_CA.ISO8859-1', 'French_Canada.1252'],
   'cs_CZ.ISO8859-2': ['cs_CZ.ISO8859-2', 'Czech_Czech Republic.1250']
 }
 for locale, alts in locales.items():
   # Note: Using alts directly in the lambda body here will bind it to the value at the
   # end of the loop. Assigning it to a default argument works around this issue.