[libc++][VE] Support futex system call on VE

[kaz7]

Summary

VE architecture doesn't distribute linux/futex.h. This patch modifies
linux implementation of atomic.cpp to to handle VE without linux/futex.h,
and to show erros if either linux/futex.h or special handling is not
implemented.

[ldionne]

Thanks for your contribution. However, it would be useful to have more information before going forward. The goal is to avoid adding random bits of complexity to the library for a system that isn't officially supported and maintained.

• What is VEOS?
• Is this part of an effort to port libc++ to that OS?
• What parts of the library are working on the OS and what parts are not working? Do you plan to subset the library using knobs like LIBCXX_ENABLE_LOCALIZATION=OFF?
• Are you able to provide CI for that OS?

Generally speaking, more information about what prompts this patch is welcome.

[kaz7]

In D94909#2504969, @ldionne wrote:

Thanks for your contribution. However, it would be useful to have more information before going forward. The goal is to avoid adding random bits of complexity to the library for a system that isn't officially supported and maintained.

• What is VEOS?
• Is this part of an effort to port libc++ to that OS?
• What parts of the library are working on the OS and what parts are not working? Do you plan to subset the library using knobs like LIBCXX_ENABLE_LOCALIZATION=OFF?
• Are you able to provide CI for that OS?

Generally speaking, more information about what prompts this patch is welcome.

Thank you for comments and sorry for missing information. VE is a NEC SX Aurora vector engine. We ported llvm/clang/libraries to VE. We are upstreaming this port following the RFC, https://lists.llvm.org/pipermail/llvm-dev/2019-April/131580.html. VE is implemented as an accelerator. Ported llvm runs on x86 and generates binary programs for VE as a cross-compiler. At the moment, VE is an experimental target. We've upstreamed llvm/clang. Now we are upstreaming libraries like builtin/libunwind/libcxx in order to run llvm-test-suite. After that, we will try to make this VE as an official target in llvm with test-suite results.

• The VEOS is an OS running on this accelerator. The VEOS is a linux subset, but almost all features are implemented.
• This is part of an effort to port not only libc++ but also clang/llvm. VE has it's own proprietary c++ compiler and c++ library. We ports llvm/clang/libc++ to provide not a proprietary compiler and libraries for VE.
• As far as I know, whole part of libc++ is working on VEOS well through libc++ test suite. Localization is working fine also.
• Yes, we will provide CI to make VE as an official target of llvm.

[kaz7]

Rebase to the latest main. VE is now an official target of llvm.
I would like to support libcxx for VE also.

[kaz7]

Change the way to handle linux/futex.h to more generic way
for ease of understanding and modifying in future.

[kaz7]

Fix typo.

[ldionne]

libcxx/src/atomic.cpp

19–21

You should be able to remove all the CMake changes and replace this by

#if __has_include(<linux/futex.h>)
#  include <linux/futex.h>
#endif

69

This is incredibly brittle and not self-documenting. Is there no way that VE can implement linux/futex.h? After all, it does provide the SYS_futex syscall and it does support FUTEX_WAIT_PRIVATE, only not with that macro name. This seems to me like a classic "workaround for something missing on the platform side", and usually it's better to round those rough edges on the platform side instead of working around them in a bunch of places (like here).

[ldionne]

Also, are you able to provide a builder that would run CI on VE?

[kaz7]

Modifies code following suggestions.

[kaz7]

In D94909#3178020, @ldionne wrote:

Also, are you able to provide a builder that would run CI on VE?

Yes. We have CI at https://lab.llvm.org/buildbot/#/builders/91. It runs builtin's tests at the moment. We will run libcxxabi tests, libcxx tests, and more soon.

libcxx/src/atomic.cpp

19–21

Thank you for suggestion. I modified code.

69

I understand how it looks like, but it is simply difficult to let OS peeople add linux/futex.h to their distribution. I tried, but it won't work. I can try it again and again...

[kaz7]

At least, I can update code by self-documenting.

[ldionne]

In D94909#3179067, @kaz7 wrote:

In D94909#3178020, @ldionne wrote:

Also, are you able to provide a builder that would run CI on VE?

Yes. We have CI at https://lab.llvm.org/buildbot/#/builders/91. It runs builtin's tests at the moment. We will run libcxxabi tests, libcxx tests, and more soon.

Libc++ uses pre-commit CI using BuildKite, as documented here: https://libcxx.llvm.org/AddingNewCIJobs.html#adding-new-ci-jobs

Can you please join our Discord Channel (https://discord.gg/jzUbyP26tQ) and reach out to me (@ldionne) there? I'll help you set it up, it's really simple. If you want, the same host will be able to run both pre-commit CI using BuildKite and the BuildBot setup, although I would recommend just running BuildKite since that's the only thing we look at.

[kaz7]

Update implementation as suggested.

[kaz7]

In D94909#3179172, @ldionne wrote:

In D94909#3179067, @kaz7 wrote:

In D94909#3178020, @ldionne wrote:

Also, are you able to provide a builder that would run CI on VE?

Yes. We have CI at https://lab.llvm.org/buildbot/#/builders/91. It runs builtin's tests at the moment. We will run libcxxabi tests, libcxx tests, and more soon.

Libc++ uses pre-commit CI using BuildKite, as documented here: https://libcxx.llvm.org/AddingNewCIJobs.html#adding-new-ci-jobs

Can you please join our Discord Channel (https://discord.gg/jzUbyP26tQ) and reach out to me (@ldionne) there? I'll help you set it up, it's really simple. If you want, the same host will be able to run both pre-commit CI using BuildKite and the BuildBot setup, although I would recommend just running BuildKite since that's the only thing we look at.

Thank you. I definitely want to do that. But, it's 1:44 AM here, Japan, and @simoll who are operating our buildbot lives in Europe. Let's organize meeting soon.

[ldionne]

In D94909#3179803, @kaz7 wrote:

Thank you. I definitely want to do that. But, it's 1:44 AM here, Japan, and @simoll who are operating our buildbot lives in Europe. Let's organize meeting soon.

No hurries at all -- sorry if that wasn't clear :-).

Just ping me whenever suits you and we can take a look together.

libcxx/src/atomic.cpp

69

Can you ask these people to come here to see what they're asking us to do?

I'm sure they would rather provide linux/futex.h than have us hardcode magic numbers in our implementation.

[kaz7]

I close this issue since the linux/futex.h for VE is available in the recent VE dstribution for RHEL8. The distribution for RHEL7 is simply not updated yet. I found that when I chatted about futex.h with colleageus in OS team. Thank you for your suggestion. ;-)

On the other hand, we also want to run pre-commit CI using BuildKite. So, we will definitely contact you about it soon. Thank you for your help.

libcxx/src/atomic.cpp

 
 #include <climits>
 #include <atomic>
 #include <functional>
 
 #ifdef __linux__
 
 #include <unistd.h>
-#include <linux/futex.h>
+#if __has_include(<linux/futex.h>)
+#  include <linux/futex.h>
+#endif
 #include <sys/syscall.h>
 
 // libc++ uses SYS_futex as a universal syscall name. However, on 32 bit architectures
 // with a 64 bit time_t, we need to specify SYS_futex_time64.
 #if !defined(SYS_futex) && defined(SYS_futex_time64)
 # define SYS_futex SYS_futex_time64
 #endif
 
 #else // <- Add other operating systems here
 
 // Baseline needs no new headers
 
 #endif
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 #ifdef __linux__
 
+#  ifdef __ve__ && !defined(FUTEX_WAIT)
+// VE doesn't distribute linux/futex.h.  Hoever, SYS_futex is implemented
+// partially as described in "Difference points for system calls",
+// https://www.hpc.nec/documents/veos/en/Difference_Point_System_Calls.pdf.
+// It describes following futex operations are supported.
+//   FUTEX_WAIT
+//   FUTEX_WAKT
+//   FUTEX_PRIVATE_FLAG
+//
+// The values of those operations are available in glibc-ve at
+// https://github.com/veos-sxarr-NEC/glibc-ve/blob/master/sysdeps/unix/sysv/linux/ve/lowlevellock.h#L30-L43.
+//
+// So, we copy defined values here as a remedy until linux/futex.h is
+// distributed.
+#    define FUTEX_WAIT         0
+#    define FUTEX_WAKE         1
+#    define FUTEX_PRIVATE_FLAG 128
+#    define FUTEX_WAIT_PRIVATE (FUTEX_WAIT | FUTEX_PRIVATE_FLAG)
+#    define FUTEX_WAKE_PRIVATE (FUTEX_WAKE | FUTEX_PRIVATE_FLAG)
+#  endif // __ve__ && !defined(FUTEX_WAIT)
+
 static void __libcpp_platform_wait_on_address(__cxx_atomic_contention_t const volatile* __ptr,
                                               __cxx_contention_t __val)
 {
     static constexpr timespec __timeout = { 2, 0 };
     syscall(SYS_futex, __ptr, FUTEX_WAIT_PRIVATE, __val, &__timeout, 0, 0);
 }
 
 static void __libcpp_platform_wake_by_address(__cxx_atomic_contention_t const volatile* __ptr,
                                               bool __notify_one)
 {
     syscall(SYS_futex, __ptr, FUTEX_WAKE_PRIVATE, __notify_one ? 1 : INT_MAX, 0, 0, 0);
 }
 
 #elif defined(__APPLE__) && defined(_LIBCPP_USE_ULOCK)
 
 extern "C" int __ulock_wait(uint32_t operation, void *addr, uint64_t value,
     uint32_t timeout); /* timeout is specified in microseconds */