[libc] Add support to compile some syscalls on 32 bit platform

[mikhail.ramalho]

Summary

This patch adds a bunch of ifdefs to handle the 32 bit versions of
some syscalls, which often only append a 64 to the name of the syscall
(with exception of SYS_lseek -> SYS_llseek and SYS_futex ->
SYS_futex_time64)

This patch also tries to handle cases where wait4 is not available
(as in riscv32): to implement wait, wait4 and waitpid when wait4 is
not available, we check for alternative wait calls and ultimately rely
on waitid to implement them all.

In riscv32, only waitid is available, so we need it to support this
platform.

[michaelrj]

libc/src/sys/wait/linux/wait.cpp

32

it might be set by the extra argument for this syscall, the man page mentions that the syscall is slightly different from the function. I've pasted the text below since I can't find it online:

The raw waitid() system call takes a fifth argument, of type struct rusage *.  If this argument  is  non-NULL,
then  it  is  used  to return resource usage information about the child, in the same manner as wait4(2).  See
getrusage(2) for details.

Even if it ends up not being relevant to this specific implementation, it'd probably still be good to put a NULL on the end of the syscall.

libc/src/sys/wait/linux/wait4.cpp

48

this list should include waitid

[mikhail.ramalho]

libc/src/sys/wait/linux/wait.cpp

32

Thanks! This actually simplifies the code!

[mikhail.ramalho]

Update waitid syscall

[mikhail.ramalho]

Added SYS_futex_time64 to syscall_numbers.h.inc

[mikhail.ramalho]

rebased

[mikhail.ramalho]

Added SYS_sched_rr_get_interval_time64 support to fix sched compilation in riscv32

[mikhail.ramalho]

ping

[michaelrj]

we should probably figure out what the appropriate waitpid errno values are before landing this. It's possible that the syscall returns them as negative numbers, but I haven't tested it yet.

libc/src/sched/linux/sched_rr_get_interval.cpp

31 ↗

(On Diff #515869)

should this return ret be here or should it instead fall down to the handling below?

also should ret be a long to match the other syscall above?

[sivachandra]

• Mostly LGTM but one suggestion: at all places where you added alternate paths to *64 syscalls, can you add commentary on when they will kick in and why they are to used?
• I will make another detailed pass over the wait* wrappers.

libc/src/__support/threads/linux/mutex.h

83

Can we add a constant, say FUTEX_SYSCALL_ID in https://github.com/llvm/llvm-project/blob/main/libc/src/__support/threads/linux/futex_word.h and avoid these widespread ifdefs?

[mikhail.ramalho]

• added siginfo_t dependency to wait.h
• added FUTEX_SYSCALL_ID
• changed SYS_sched_rr_get_interval_time64 to use struct __kernel_timespec instead of a long arr[2]
• improve wait* documentation

[mikhail.ramalho]

Apply clang-format to syscall_numbers.h.inc

[sivachandra]

libc/src/__support/File/linux_file.cpp

72

Can you point to some documentation about llseek on rv32?

libc/src/__support/threads/linux/futex_word.h

23 ↗

(On Diff #517369)

Quick googling tells me that SYS_futex_time64 is rv32 only construct. Can you add comment here saying that?

Also, WDYT doing it this way?

#if <rv32>
#ifndef SYS_futex_time64
#error "..."
#endif
constexpr auto FUTEX_SYSCALL_ID = SYS_futex_time64;
#else
constexpr auto FUTEX_SYSCALL_ID = SYS_futex;
#endif

libc/src/sched/linux/sched_rr_get_interval.cpp

31 ↗

(On Diff #517369)

ret > 0 ?

libc/src/sys/sendfile/linux/sendfile.cpp

26

Do you mean that the offset should be a pointer to a 64-bit value? If yes, can you add a static_assert here?

static_assert(sizeof(off_t) == 64);

libc/src/sys/wait/linux/wait4.cpp

26

Which one is required for rv32, SYS_waitpid or SYS_waitid?

31

Where is this getrusage coming from?

38

to

44

Why do we need this assert?

libc/src/sys/wait/linux/waitpid.cpp

22

Same question: can we limit to the exact SYS_wait* we want for rv32?

37

Why do we need this assert?

libc/src/unistd/linux/dup2.cpp

33

Can you add comments about how this 64 suffixed syscall number differs from the non-suffixed syscall?

libc/src/unistd/linux/ftruncate.cpp

24

Ditto.

libc/src/unistd/linux/truncate.cpp

24

Ditto.

[mikhail.ramalho]

• Added comments about sys_*64 syscalls
• Added assertion about offset size in sys_*64 syscalls
• Fixed wait4 implementation
• Rewrote wait and waitpid to use wait4

libc/src/__support/File/linux_file.cpp

72

You can find the llseek documentation here: https://man7.org/linux/man-pages/man3/lseek64.3.html

libc/src/__support/threads/linux/futex_word.h

23 ↗

(On Diff #517369)

AFAIK, SYS_futex_time64 is defined in every 32-bit arch that uses 64-bit time, e.g., in my armv7 machine:

$ clang -dM -E main1.c | grep futex
#define SYS_futex __NR_futex
#define SYS_futex_time64 __NR_futex_time64
#define __NR_futex (__NR_SYSCALL_BASE + 240)
#define __NR_futex_time64 (__NR_SYSCALL_BASE + 422)

The case for rv32 is that is only has SYS_futex_time64.

libc/src/sys/wait/linux/wait4.cpp

26

SYS_waitid

44

Fixed.

[mikhail.ramalho]

ping

[michaelrj]

libc/src/sys/wait/linux/wait.cpp

46

we generally don't allow calling public libc functions from other libc functions, so wait4 should be moved to a shared internal function. This allows platforms to include functions without having to deal with dependency chains. You can see how this is generally done with the string to integer functions that all include strtointeger, though in this case it probably doesn't need to go in __support.

libc/src/sys/wait/linux/wait4.cpp

46

We generally try to avoid setting errno in internal functions, so if you move wait4 to be an internal function you'll probably have to change its return type to ErrorOr<pid_t> and set errno after the call. This will be similar to how fseek works..

[sivachandra]

This is good to go after addressing @michaelrj's comments.

libc/src/sys/wait/linux/wait4.cpp

43

I think you should #include <signal.h> for CLD_* macros?

[mikhail.ramalho]

• Address comment about wait4 implementation
• Fix sched_getscheduler return value
• Fix sprintf test case for 32-bit systems

[mikhail.ramalho]

This PR is not done yet as I'm getting a weird error on the chmod test on rv32:

FAILED: projects/libc/test/src/sys/stat/CMakeFiles/libc.test.src.sys.stat.chmod_test /home/root/llvm-project/build/projects/libc/test/src/sys/stat/CMakeFiles/libc.test.src.sys.stat.chmod_test 
cd /home/root/llvm-project/build/projects/libc/test/src/sys/stat && /home/root/llvm-project/build/projects/libc/test/src/sys/stat/libc.test.src.sys.stat.chmod_test.__build__
[ RUN      ] LlvmLibcChmodTest.ChangeAndOpen
/home/root/llvm-project/libc/test/src/sys/stat/chmod_test.cpp:46: FAILURE
      Expected: __llvm_libc::open(TEST_FILE, 02000 | 01)
      Which is: 3
To be equal to: -1
      Which is: -1
/home/root/llvm-project/libc/test/src/sys/stat/chmod_test.cpp:47: FAILURE
          Expected: __llvm_libc::__llvmlibc_internal_errno
          Which is: 0
To be not equal to: 0
          Which is: 0

Apparently, I'm able to set the file as read-only but can also open it in write mode.

[mikhail.ramalho]

• Fixed ftruncate, truncate, pwrite and pread syscalls
• Rebased

[mikhail.ramalho]

Removed wrong ifdef

[mikhail.ramalho]

Changed wait4impl to return an ErrorOr object

[mikhail.ramalho]

folks, can I get another round of reviews on this patch? I've addressed all the comments.

[michaelrj]

LGTM with a minor nit

libc/test/src/sched/sched_rr_get_interval_test.cpp

32 ↗

(On Diff #546523)

nit: Generally I'd recommend using either long long or uint64_t for a larger type. long is only required to be at least 32 bits, and some systems (notably windows) use that size.

[gchatelet]

Hello Mikhail, it looks like https://reviews.llvm.org/D148371 broke the x86 linux build bots.
https://lab.llvm.org/buildbot/#/grid?tag=libc
https://lab.llvm.org/buildbot/#/builders/78/builds/55807/steps/4/logs/stdio

/home/sivachandra/libc-x86_64-debian/libc-x86_64-debian/llvm-project/libc/src/sched/linux/sched_rr_get_interval.cpp:15:10: fatal error: 'linux/time_types.h' file not found
#include <linux/time_types.h> // For __kernel_timespec.

Can you revert and reland when it's fixed? Thx!

[gchatelet]

Just saw https://github.com/llvm/llvm-project/commit/29dec352124a5fc1dcf03a640c2621496651ee90 which should fix it :)

libc/src/__support/File/linux_file.cpp

 }
 
 ErrorOr<long> seek_func(File *f, long offset, int whence) {
   auto *lf = reinterpret_cast<LinuxFile *>(f);
   long result;
 #ifdef SYS_lseek
   int ret = __llvm_libc::syscall_impl(SYS_lseek, lf->get_fd(), offset, whence);
   result = ret;
+#elif defined(SYS_llseek)
+  long ret =
+      __llvm_libc::syscall_impl(SYS_llseek, lf->get_fd(), offset, whence);
+  result = ret;
 #elif defined(SYS__llseek)
   long result;
   int ret = __llvm_libc::syscall_impl(SYS__llseek, lf->get_fd(), offset >> 32,
                                       offset, &result, whence);
 #else
-#error "lseek and _llseek syscalls not available to perform a seek operation."
+#error "lseek, llseek and _llseek syscalls not available."
 #endif
 
   if (ret < 0)
     return Error(-ret);
 
   return result;
 }
 

libc/src/__support/threads/linux/mutex.h

while (true) {
       }
 
       switch (LockState(mutex_status)) {
       case LockState::Waiting:
         // If other threads are waiting already, then join them. Note that the
         // futex syscall will block if the futex data is still
         // `LockState::Waiting` (the 4th argument to the syscall function
         // below.)
+#if SYS_futex
         __llvm_libc::syscall_impl(SYS_futex, &futex_word.val,
                                   FUTEX_WAIT_PRIVATE,
                                   FutexWordType(LockState::Waiting), 0, 0, 0);
+#elif defined(SYS_futex_time64)
+        __llvm_libc::syscall_impl(SYS_futex_time64, &futex_word.val,
+                                  FUTEX_WAIT_PRIVATE,
+                                  FutexWordType(LockState::Waiting), 0, 0, 0);
+#else
+#error "futex and futex_time64 syscalls not available."
+#endif
         was_waiting = true;
         // Once woken up/unblocked, try everything all over.
         continue;
       case LockState::Locked:
         // Mutex has been locked by another thread so set the status to
         // LockState::Waiting.
         if (futex_word.compare_exchange_strong(
                 locked_status, FutexWordType(LockState::Waiting))) {
           // If we are able to set the futex data to `LockState::Waiting`, then
           // we will wait for the futex to be woken up. Note again that the
           // following syscall will block only if the futex data is still
           // `LockState::Waiting`.
+#if SYS_futex
           __llvm_libc::syscall_impl(SYS_futex, &futex_word, FUTEX_WAIT_PRIVATE,
                                     FutexWordType(LockState::Waiting), 0, 0, 0);
+#elif defined(SYS_futex_time64)
+          __llvm_libc::syscall_impl(SYS_futex_time64, &futex_word,
+                                    FUTEX_WAIT_PRIVATE,
+                                    FutexWordType(LockState::Waiting), 0, 0, 0);
+#else
+#error "futex and futex_time64 syscalls not available."
+#endif
           was_waiting = true;
         }
         continue;
       case LockState::Free:
         // If it was LockState::Free, we shouldn't be here at all.
         return MutexError::BAD_LOCK_STATE;
       }
     }
   }
 
   MutexError unlock() {
     while (true) {
       FutexWordType mutex_status = FutexWordType(LockState::Waiting);
       if (futex_word.compare_exchange_strong(mutex_status,
                                              FutexWordType(LockState::Free))) {
         // If any thread is waiting to be woken up, then do it.
+#if SYS_futex
         __llvm_libc::syscall_impl(SYS_futex, &futex_word, FUTEX_WAKE_PRIVATE, 1,
                                   0, 0, 0);
+#elif defined(SYS_futex_time64)
+        __llvm_libc::syscall_impl(SYS_futex_time64, &futex_word,
+                                  FUTEX_WAKE_PRIVATE, 1, 0, 0, 0);
+#else
+#error "futex and futex_time64 syscalls not available."
+#endif
         return MutexError::NONE;
       }
 
       if (mutex_status == FutexWordType(LockState::Locked)) {
         // If nobody was waiting at this point, just free it.
         if (futex_word.compare_exchange_strong(mutex_status,
                                                FutexWordType(LockState::Free)))
           return MutexError::NONE;

libc/src/sys/sendfile/linux/sendfile.cpp

 #include "src/errno/libc_errno.h"
 #include <sys/sendfile.h>
 #include <sys/syscall.h> // For syscall numbers.
 
 namespace __llvm_libc {
 
 LLVM_LIBC_FUNCTION(ssize_t, sendfile,
                    (int out_fd, int in_fd, off_t *offset, size_t count)) {
+#ifdef SYS_sendfile
   long ret =
       __llvm_libc::syscall_impl(SYS_sendfile, in_fd, out_fd, offset, count);
+#elif defined(SYS_sendfile64)
+  long ret =
+      __llvm_libc::syscall_impl(SYS_sendfile64, in_fd, out_fd, offset, count);
+#else
+#error "sendfile and sendfile64 syscalls not available."
+#endif
   if (ret < 0) {
     libc_errno = -ret;
     return -1;
   }
   return ret;
 }
 
 } // namespace __llvm_libc

libc/src/sys/wait/linux/wait.cpp

 //===-- Linux implementation of wait --------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "src/sys/wait/wait.h"
 
 #include "src/__support/OSUtil/syscall.h" // For internal syscall function.
 #include "src/__support/common.h"
+#include "src/__support/libc_assert.h"
 
 #include "src/errno/libc_errno.h"
 #include <sys/syscall.h> // For syscall numbers.
 #include <sys/wait.h>
 
 namespace __llvm_libc {
 
 // The implementation of wait here is very minimal. We will add more
 // functionality and standard compliance in future.
 
 LLVM_LIBC_FUNCTION(pid_t, wait, (int *wait_status)) {
+#if SYS_wait4
   pid_t pid = __llvm_libc::syscall_impl(SYS_wait4, -1, wait_status, 0, 0);
+#elif defined(SYS_waitpid)
+  pid_t pid = __llvm_libc::syscall_impl(SYS_waitpid, -1, wait_status, 0);
+#elif defined(SYS_waitid)
+  siginfo_t info;
+  if (__llvm_libc::syscall_impl(SYS_waitid, P_ALL, 0, &info, WEXITED, 0) == -1)
+    // TODO: what's the libc_errno here?
+    return -1;
+  LIBC_ASSERT(wait_status != nullptr);
+  *wait_status = info.si_status;
+  pid_t pid = info.si_pid;
+#else
+#error "waitpid, wait4 and waitid syscalls not available."
+#endif
   if (pid < 0) {
     // Error case, a child process was not created.
     libc_errno = -pid;
     return -1;
   }
 
   return pid;
 }
 
 } // namespace __llvm_libc

libc/src/sys/wait/linux/wait4.cpp

 //===-- Linux implementation of wait4 -------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "src/sys/wait/wait4.h"
 
 #include "src/__support/OSUtil/syscall.h" // For internal syscall function.
 #include "src/__support/common.h"
+#include "src/__support/libc_assert.h"
 
 #include "src/errno/libc_errno.h"
 #include <sys/syscall.h> // For syscall numbers.
 #include <sys/wait.h>
 
 namespace __llvm_libc {
 
 LLVM_LIBC_FUNCTION(pid_t, wait4,
                    (pid_t pid, int *wait_status, int options,
                     struct rusage *usage)) {
+#if SYS_wait4
   pid = __llvm_libc::syscall_impl(SYS_wait4, pid, wait_status, options, usage);
+#elif defined(SYS_waitpid)
+  pid_t pid = __llvm_libc::syscall_impl(SYS_waitpid, pid, wait_status, options);
+  if (pid > 0) {
+    if (usage != nullptr && (getrusage(info.si_pid, usage) == -1))
+      // TODO: what's the libc_errno here?
+      return -1;
+  }
+#elif defined(SYS_waitid)
+  siginfo_t info;
+  if (__llvm_libc::syscall_impl(SYS_waitid, P_PID, pid, &info,
+                                options | WEXITED | WSTOPPED | WCONTINUED,
+                                usage) == -1)
+    return -1;
+  LIBC_ASSERT(wait_status != nullptr);
+  *wait_status = info.si_status;
+  pid = info.si_pid;
+#else
+#error "waitpid, wait4 and waitid syscalls not available."
+#endif
   if (pid < 0) {
     libc_errno = -pid;
     return -1;
   }
   return pid;
 }
 
 } // namespace __llvm_libc

libc/src/sys/wait/linux/waitpid.cpp

 //===-- Linux implementation of waitpid -----------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "src/sys/wait/waitpid.h"
 
 #include "src/__support/OSUtil/syscall.h" // For internal syscall function.
 #include "src/__support/common.h"
+#include "src/__support/libc_assert.h"
 
 #include "src/errno/libc_errno.h"
 #include <sys/syscall.h> // For syscall numbers.
 #include <sys/wait.h>
 
 namespace __llvm_libc {
 
 LLVM_LIBC_FUNCTION(pid_t, waitpid, (pid_t pid, int *wait_status, int options)) {
+#if SYS_waitpid
+  pid = __llvm_libc::syscall_impl(SYS_waitpid, pid, wait_status, options);
+#elif defined(SYS_wait4)
   pid = __llvm_libc::syscall_impl(SYS_wait4, pid, wait_status, options, 0);
+#elif defined(SYS_waitid)
+  siginfo_t info;
+  if (__llvm_libc::syscall_impl(SYS_waitid, P_PID, pid, &info, options, 0) ==
+      -1)
+    // TODO: what's the libc_errno here?
+    return -1;
+  LIBC_ASSERT(wait_status != nullptr);
+  *wait_status = info.si_status;
+  pid = info.si_pid;
+#else
+#error "waitpid, wait4 and waitid syscalls not available."
+#endif
   if (pid < 0) {
     libc_errno = -pid;
     return -1;
   }
   return pid;
 }
 
 } // namespace __llvm_libc

libc/src/unistd/linux/dup2.cpp

#ifdef SYS_dup2
   // If dup2 syscall is available, we make use of directly.
   long ret = __llvm_libc::syscall_impl(SYS_dup2, oldfd, newfd);
 #elif defined(SYS_dup3)
   // If dup2 syscall is not available, we try using the dup3 syscall. However,
   // dup3 fails if oldfd is the same as newfd. So, we handle that case
   // separately before making the dup3 syscall.
   if (oldfd == newfd) {
     // Check if oldfd is actually a valid file descriptor.
+#if SYS_fcntl
     long ret = __llvm_libc::syscall_impl(SYS_fcntl, oldfd, F_GETFD);
+#elif defined(SYS_fcntl64)
+    long ret = __llvm_libc::syscall_impl(SYS_fcntl64, oldfd, F_GETFD);
+#else
+#error "SYS_fcntl and SYS_fcntl64 syscalls not available."
+#endif
     if (ret >= 0)
       return oldfd;
     libc_errno = -ret;
     return -1;
   }
   long ret = __llvm_libc::syscall_impl(SYS_dup3, oldfd, newfd, 0);
 #else
 #error "dup2 and dup3 syscalls not available."

libc/src/unistd/linux/ftruncate.cpp

 
 #include "src/errno/libc_errno.h"
 #include <sys/syscall.h> // For syscall numbers.
 #include <unistd.h>
 
 namespace __llvm_libc {
 
 LLVM_LIBC_FUNCTION(int, ftruncate, (int fd, off_t len)) {
+#ifdef SYS_ftruncate
   int ret = __llvm_libc::syscall_impl(SYS_ftruncate, fd, len);
+#elif defined(SYS_ftruncate64)
+  int ret = __llvm_libc::syscall_impl(SYS_ftruncate64, fd, len);
+#else
+#error "ftruncate and ftruncate64 syscalls not available."
+#endif
+
   if (ret < 0) {
     libc_errno = -ret;
     return -1;
   }
   return 0;
 }
 
 } // namespace __llvm_libc

libc/src/unistd/linux/lseek.cpp

 
 namespace __llvm_libc {
 
 LLVM_LIBC_FUNCTION(off_t, lseek, (int fd, off_t offset, int whence)) {
   off_t result;
 #ifdef SYS_lseek
   long ret = __llvm_libc::syscall_impl(SYS_lseek, fd, offset, whence);
   result = ret;
+#elif defined(SYS_llseek)
+  long ret = __llvm_libc::syscall_impl(SYS_llseek, fd, offset, whence);
+  result = ret;
 #elif defined(SYS__llseek)
-  long ret = __llvm_libc::syscall_impl(SYS__llseek, fd, offset >> 32, offset,
-                                       &result, whence);
+  int ret = __llvm_libc::syscall_impl(SYS__llseek, fd, offset >> 32, offset,
 #else
-#error "lseek and _llseek syscalls not available."
+#error "lseek, llseek and _llseek syscalls not available."
 #endif
 
   if (ret < 0) {
     libc_errno = -ret;
     return -1;
   }
   return result;
 }
 
 } // namespace __llvm_libc

libc/src/unistd/linux/truncate.cpp

 #include "src/errno/libc_errno.h"
 
 #include <sys/syscall.h> // For syscall numbers.
 #include <unistd.h>
 
 namespace __llvm_libc {
 
 LLVM_LIBC_FUNCTION(int, truncate, (const char *path, off_t len)) {
+#ifdef SYS_truncate
   int ret = __llvm_libc::syscall_impl(SYS_truncate, path, len);
+#elif defined(SYS_truncate64)
+  int ret = __llvm_libc::syscall_impl(SYS_truncate64, path, len);
+#else
+#error "truncate and truncate64 syscalls not available."
+#endif
   if (ret < 0) {
     libc_errno = -ret;
     return -1;
   }
   return 0;
 }
 
 } // namespace __llvm_libc