[TSan] Add SystemZ support

[iii]

Summary

This patch series implements ThreadSanitizer support for SystemZ.
Patches 1-3 make the necessary common code adjustments, patches 4-7
implement various bits, and patch 8 actually enables it. golang
support is currently missing and will be submitted separately.

[dvyukov]

Some solid work here.

I only have a question re mmap handling.

compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp

781 ↗

(On Diff #357214)

TSan runtime is supposed to mprotect all ranges that are inaccessible to the application. How does it happen that kernel still returns such an address? Do we fail to mprotect all ranges on s390? If so should that be fixed instead?
I am concerned that if kernel can return unsupported addresses, it probably can returns them very early before exhausting all supported addresses (e.g. always return unsupported from the start).
Or is it that we mprotect everything, but kernel still somehow stomps on it? If so can this address overlap with, say, tsan shadow? If yes, them the munmap will unmap our shadow which is not good.

[iii]

• Reserve the address space "tail" in the "[TSan] Define C/C++ address ranges for SystemZ" patch.
• Drop the "[TSan] Simulate OOM in mmap_interceptor()" patch.
• Group "[TSan] Use zeroext for function parameters" with the common code patches.

[iii]

• Fix a local variable naming issue CheckAndProtect().

[iii]

compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp

781 ↗

(On Diff #357214)

The common code in CheckAndProtect() didn't mprotect the address space "tail" - I've added a special case for that.

[iii]

Regarding the openmp test failures: I tried check-openmp on a x86_64 machine, and they occur both with and without this patch series, so must be unrelated.

[uweigand]

See inline comments ... otherwise the SystemZ platform-specific parts look good to me.

compiler-rt/lib/tsan/rtl/tsan_platform_posix.cpp

123 ↗

(On Diff #357889)

Did you test this on older kernels without 5-level page table support? I believe the allocation / mprotect may fail on those ...

compiler-rt/lib/tsan/rtl/tsan_rtl_s390x.S

22 ↗

(On Diff #357889)

Do we need CFI for r2/r3 ? Those are call-clobbered any cannot be unwound normally anyway ...

[iii]

compiler-rt/lib/tsan/rtl/tsan_platform_posix.cpp

123 ↗

(On Diff #357889)

No, not really. Would it make sense to probe here? E.g. first try 0xfffffffffffff000, then 0x20000000000000. Or is there a way to query user_addr_max() / TASK_SIZE_MAX / TASK_SIZE?

compiler-rt/lib/tsan/rtl/tsan_rtl_s390x.S

22 ↗

(On Diff #357889)

I'm not quite sure, but glibc does this (e.g. in sysdeps/s390/s390-64/dl-trampoline.h), so I figured I'll do this here as well just in case.

[uweigand]

compiler-rt/lib/tsan/rtl/tsan_platform_posix.cpp

123 ↗

(On Diff #357889)

I don't know of any way to query this. You can simply do the allocation in stages (first to the top of the three-pagetable range, then four, then five), and ignore failures. As an unfortunate side effect this will force the kernel to allocate five levels of page tables even when unnecessary, but I don't think there's anything we can do about that.

compiler-rt/lib/tsan/rtl/tsan_rtl_s390x.S

22 ↗

(On Diff #357889)

Huh. Well I guess it doesn't hurt either way.

[iii]

compiler-rt/lib/tsan/rtl/tsan_platform_posix.cpp

123 ↗

(On Diff #357889)

3-level page table limit is quite below the application range defined in this series (0x40000000000 < 0xc00000000000). Are there any relevant kernels that do not support 4-level page tables? I tried to search the git history, and it looks as if even 2.6 ones have it.

[uweigand]

compiler-rt/lib/tsan/rtl/tsan_platform_posix.cpp

123 ↗

(On Diff #357889)

Ah, right. Yes, we can certainly assume 4-level support at this point.

[iii]

Based on Ulrich's feedback I tested the series on RHEL7, which, in
addition to an old kernel, contains an old glibc and an old toolchain.
This uncovered a few extra issues, which are fixed here:

• Call __tls_get_offset() in order to force TLS allocation.
• Align thread_registry_placeholder.
• Protect the address space "tail" in two steps: until the 4-level page table boundary (this must succeed) and then until the 5-level page table boundary (this may fail).

Also, the Go support is finally working
(https://github.com/iii-i/go/tree/tsan-s390x), so, since we need to
make another round anyway, I've included the LLVM part here as well.

[iii]

• Fix style issues (seems like arc diff is linting only the top commit?).

[uweigand]

The SystemZ specific changes all look good to me now, thanks!

compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp

 #if SANITIZER_LINUX
 void internal_sigdelset(__sanitizer_sigset_t *set, int signum) {
   signum -= 1;
   CHECK_GE(signum, 0);
   CHECK_LT(signum, sizeof(*set) * 8);
   __sanitizer_kernel_sigset_t *k_set = (__sanitizer_kernel_sigset_t *)set;
   const uptr idx = signum / (sizeof(k_set->sig[0]) * 8);
   const uptr bit = signum % (sizeof(k_set->sig[0]) * 8);
-  k_set->sig[idx] &= ~(1 << bit);
+  k_set->sig[idx] &= ~((uptr)1 << bit);
 }
 
 bool internal_sigismember(__sanitizer_sigset_t *set, int signum) {
   signum -= 1;
   CHECK_GE(signum, 0);
   CHECK_LT(signum, sizeof(*set) * 8);
   __sanitizer_kernel_sigset_t *k_set = (__sanitizer_kernel_sigset_t *)set;
   const uptr idx = signum / (sizeof(k_set->sig[0]) * 8);
   const uptr bit = signum % (sizeof(k_set->sig[0]) * 8);
-  return k_set->sig[idx] & (1 << bit);
+  return k_set->sig[idx] & ((uptr)1 << bit);
 }
 #elif SANITIZER_FREEBSD
 void internal_sigdelset(__sanitizer_sigset_t *set, int signum) {
   sigset_t *rset = reinterpret_cast<sigset_t *>(set);
   sigdelset(rset, signum);
 }
 
 bool internal_sigismember(__sanitizer_sigset_t *set, int signum) {

compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.h

 #endif
 #if defined(__s390x__)
   __sanitizer_sigset_t sa_mask;
 #endif
 };
 #endif // !SANITIZER_ANDROID
 
 #if defined(__mips__)
-struct __sanitizer_kernel_sigset_t {
-  uptr sig[2];
-};
+#define __SANITIZER_KERNEL_NSIG 128
 #else
+#define __SANITIZER_KERNEL_NSIG 64
+#endif
+
 struct __sanitizer_kernel_sigset_t {
-  u8 sig[8];
+  uptr sig[__SANITIZER_KERNEL_NSIG / (sizeof(uptr) * 8)];
 };
-#endif
 
 // Linux system headers define the 'sa_handler' and 'sa_sigaction' macros.
 #if SANITIZER_MIPS
 struct __sanitizer_kernel_sigaction_t {
   unsigned int sa_flags;
   union {
     void (*handler)(int signo);
     void (*sigaction)(int signo, __sanitizer_siginfo *info, void *ctx);