[TSan][MIPS] Adding support for MIPS64

[slthakur]

Summary

patch dependent on http://reviews.llvm.org/D7013

[kcc]

I will let Dmitry review this in detail, but my general request is to avoid #ifdefs inside the code as much as possible.
Please try to move asm many #ifdefs as you can into headers and then use plain if() in the code.
For example, you can do if(SANITIZER_CAN_USE_ALLOCATOR64) instead of #ifdef SANITIZER_CAN_USE_ALLOCATOR64

[slthakur]

Checked if SIZEOF_INT128 is defined for 128 bit atomic operations in tsan_interface_atomic.cc

[samsonov]

lib/tsan/CMakeLists.txt

103 ↗

(On Diff #16379)

Please merge this into x86_64 case somehow.

lib/tsan/rtl/tsan_interceptors.cc

44 ↗

(On Diff #16379)

acquired (here and below)

661 ↗

(On Diff #16379)

I don't understand this comment. Why does REAL(pthread_create) crash?

lib/tsan/rtl/tsan_interface_atomic.cc

43 ↗

(On Diff #16379)

use
#if !defined(TSAN_GO) && defined(SIZEOF_INT128)

Or, better, fix this macro definition in Clang.

lib/tsan/rtl/tsan_platform_linux.cc

318 ↗

(On Diff #16379)

Please reduce the amount of #ifs here (e.g. introduce a helper function).

[slthakur]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

As per my observation, in x86_64 first cxa_guard_aquired() is called and its interceptor calls Initialize().
Initialize() creates background thread.
But in mips64 interceptor_memset gets called before cxa_guard_aquire() interceptor. So Initialize() is called from within intercerptor_memset and REAL(pthread_create) emits a SIGIOT.

@dvyukov I don't know why such behaviour in mips, can you help us with this?

[samsonov]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

Why does the call to REAL(pthread_create) succeeds if it is issued from __cxa_guard_acquired interceptor and fails if it is issued from memset interceptor? I think you'd have to figure it out.

I don't see the immediate problem from your description: memset interceptor calls Initialize(), which calls InitializeIntercepotrs() (that is supposed to initialize all REAL(foo) function pointers), and only then calls internal_start_thread.

[slthakur]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

While debugging the code, I observed that interceptor_memset and cxa_guard_acquire are being called even before __start. So its reference is probably in LLVM/clang. I have no knowledge from where they are being called.

Through strace in x86_64, the sequence of calls observed:

rt_sigaction() -> rt_sigprocmask() -> __cxa_guard_acquired() -> Initialize()

In case of mips64, the sequence is:

__interceptor_memset() -> Initialize()

So for mips64 there is no signal handler registered during Initialize() because rt_sigaction syscall is not yet issued.

I will be very thankful if someone could advice me on how to solve this issue.

[samsonov]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

It's hard to predict which TSan interceptor will be called first (and thus will be first to call __tsan::Initialize()). For example, on my x86_64 host the stack trace is:

__tsan::Initialize
ScopedInterceptor::ScopedInterceptor
__cxa_guard_acquire
__future_category_instance
__static_initialization_and_destruction_0
_GLOBAL__sub_I_compatibility_thread_c__0x.cc
call_init
_dl_start_user

I don't see why signal handler is at all important. Can you just debug your TSan-instrumented program in gdb, reach the place where it fails with SIGIOT/SIGABRT and learn the reason why it's happening? E.g. it calls REAL(foo), but this function pointer is not yet initialized. Or it accesses mprotect()-ed memory. Or something else.

Sorry, I can't yet diagnose the issue from your descriptions.

[slthakur]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

Hi,

The intercepted memset is called during pthread initialization function __pthread_initialize_minimal_internal ().

Sequence of calls for MIPS:

__pthread_initialize_minimal_internal () -> __sigemptyset () -> __interceptor_memset () -> pthread_create () -> __GI_abort ()

The intercepted memset calls pthread_create() before the minimal initialization required for pthread_create is complete and this results in abort.
On x86, __pthread_initialize_minimal_internal () does not call memset and thus can complete the minimal initialization.
For MIPS, we need to avoid calling pthread_create () if memset is called during pthread initialization.

[samsonov]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

So, this has nothing to do with __cxa_guard_acquired or memset. You are just not allowed to call pthread_create() from TSan runtime before __pthread_initialize_minimal_internal exits. For example, you can try to intercept __pthread_initialize_minimal_internal and use ScopedIgnoreInterceptors to disable TSan interceptors for sigemptyset(), memset() or whatever.

[slthakur]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

Hi,

I tried to intercept __pthread_initialize_minimal_internal (), but it does not get intercepted. As far as I know, this is probably because references to __pthread_initialize_minimal_internal () are resolved when glibc is compiled. I also tried to search for calls to functions occurring before call to __pthread_initialize_minimal_internal () which would be resolved at runtime, so that we could intercept them and use ScopedIgnoreInterceptors, but I couldn’t see any such function call.

[samsonov]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

Who calls __pthread_initialize_minimal_internal at all in that case? Is it called from library constructor? As a quick workaround, you can disable background thread in MIPS port - IIRC it's not critical to TSan functioning.

[slthakur]

lib/tsan/rtl/tsan_interceptors.cc

661 ↗

(On Diff #16379)

Who calls __pthread_initialize_minimal_internal at all in that case?

Here is the backtrace at __pthread_initialize_minimal_internal () :

(gdb) bt
#0  __pthread_initialize_minimal_internal () at nptl-init.c:281
#1  0x000000ffed9567e4 in _init () at ../ports/sysdeps/mips/mips64/n64/crti.S:80
#2  0x000000ffed9a0d94 in call_init (l=0xffed9c0930, argc=argc@entry=1, argv=argv@entry=0xffff77a658, env=env@entry=0xffff77a668)
    at dl-init.c:64
#3  0x000000ffed9a0fc0 in call_init (env=0xffff77a668, argv=0xffff77a658, argc=1, l=<optimized out>) at dl-init.c:36
#4  _dl_init (main_map=0xffed9c3848, argc=<optimized out>, argv=0xffff77a658, env=0xffff77a668) at dl-init.c:93
#5  0x000000ffed9906e0 in _dl_start_user () from /lib64/ld.so.1

Is it called from library constructor?

As per my observation, call to __pthread_initialize_minimal_internal () occurs even before preinit functions execute.
And as fas as I know library constructors execute after preinit functions execute.

[slthakur]

Updated patch as per suggestions of Samsonov.

[slthakur]

Removed extra empty line in tsan_rtl.cc

[samsonov]

What is the status of check-tsan command with these changes applied?

lib/tsan/CMakeLists.txt

80 ↗

(On Diff #17438)

set TSAN_ASM_SOURCES to empty list for non-x86_64 architectures.

lib/tsan/rtl/tsan_interface_atomic.cc

36 ↗

(On Diff #17438)

This change is wrong - it won't work if you build TSan runtime with GCC. Why do you need it?

129 ↗

(On Diff #17438)

Shouldn't this be related to __TSAN_HAS_INT128 ?

lib/tsan/rtl/tsan_platform.h

30 ↗

(On Diff #17438)

Care to add a similar comment for MIPS?

lib/tsan/rtl/tsan_platform_linux.cc

218 ↗

(On Diff #17438)

Where do these magic numbers come from? Please use a named constant for them.

295 ↗

(On Diff #17438)

factor this into a separate constant.

lib/tsan/rtl/tsan_rtl.cc

1001 ↗

(On Diff #17438)

Looks like it should be inline function in the same file where you define the memory layout.

[slthakur]

Updated patch as per suggestions by Samsonov

[slthakur]

What is the status of check-tsan command with these changes applied?

Expected Passes    : 159
Expected Failures  : 1
Unsupported Tests  : 1
Unexpected Failures: 15

lib/tsan/rtl/tsan_interface_atomic.cc

36 ↗

(On Diff #17438)

We have not yet implemented 128-bit operations for mips as of clang version 3.6.0. We need to disable __TSAN_HAS_INT128 because tsan fails to compile with the use of 128-bit atomic operations. I have changed this to disable __TSAN_HAS_INT128 only for mips64.

lib/tsan/rtl/tsan_platform_linux.cc

218 ↗

(On Diff #17438)

If I understand correctly this address range is part of high application memory which is used for thread stack and large user mmaps.

[samsonov]

OK, this patch is getting closer to commit-ready state. However, I'd still ask Dmitry to review / sign this off.

What is the main reason of unexpected failures in test?

lib/tsan/CMakeLists.txt

86 ↗

(On Diff #17590)

set(TSAN_ASM_SOURCES)

lib/tsan/rtl/tsan_platform.h

135 ↗

(On Diff #17590)

No, please don't copy-paste this code around. You can move this function to tsan_rtl.h, where allocator is defined.

lib/tsan/rtl/tsan_rtl.cc

331 ↗

(On Diff #17590)

You can shorten the comment and tell that on MIPS TSan initialization is run before __pthread_initialize_minimal_internal() is finished, so we can't spawn new threads.

[slthakur]

What is the main reason of unexpected failures in test?

Failing Tests (15):

ThreadSanitizer :: global_race.cc
ThreadSanitizer :: global_race2.cc
ThreadSanitizer :: global_race3.cc
ThreadSanitizer :: longjmp.cc
ThreadSanitizer :: longjmp2.cc
ThreadSanitizer :: longjmp3.cc
ThreadSanitizer :: longjmp4.cc
ThreadSanitizer :: map32bit.cc
ThreadSanitizer :: mmap_large.cc
ThreadSanitizer :: race_on_mutex.c
ThreadSanitizer :: signal_longjmp.cc
ThreadSanitizer :: signal_errno.cc
ThreadSanitizer :: stack_sync_reuse.cc
ThreadSanitizer :: tls_race.cc
ThreadSanitizer :: tls_race2.cc

global_race.cc, global_race2.cc, global_race3.cc : File Check expects a 47-bit address in place of ADDR, ADDR2, ADDR3 in the test cases. Where as MIPS64 uses 40-bit addresses.

longjmp.cc, longjmp2.cc, longjmp3.cc, longjmp4.cc, signal_longjmp.cc : We have not implemented longjmp/setjmp assemble for MIPS.

map32bit.cc : MAP_32BIT flag for mmap is supported only for x86_64.

mmap_large.cc : Mmap start address is given according to 47-bit address space. Where as MIPS64 uses 40-bit addresses.

race_on_mutex.c : Stack trace is incorrect because of interception of memset call from nptl/pthread_mutex_init.c:91 in glibc.

signal_errno.cc : Stack trace is incorrect because pc is incremented by 1 at tsan_interceptors.cc:1908 in CallUserSignalHandler (). Here, we need to use StackTrace::GetNextInstructionPC () in place of pc +1.

stack_sync_reuse.cc : There is address mismatch between addr and &s in the test case.

tls_race.cc : Test case passes on application of http://reviews.llvm.org/D5616 patch because this patch provides the definition of ThreadDescriptorSize (), GetTls () and ThreadSelf () for MIPS.

tls_race2.cc : ReportSourceLocation is Stack instead of TLS. This is because there is an overlap between TLS and Stack address range.

[slthakur]

Updated patch as per suggestions by Samsonov

[samsonov]

LGTM. However, please let Dmitry sign this off.

lib/tsan/rtl/tsan_platform_linux.cc

217 ↗

(On Diff #17625)

Mark these as const

[slthakur]

Marked kMadviseRangeSize and kMadviseRangeBeg as const

[dvyukov]

LGTM with nits

lib/tsan/rtl/tsan_interceptors.cc

43 ↗

(On Diff #17756)

In mips linux kernel:

#define SIGRTMAX _NSIG
#define _NSIG 128

So signal 128 is a valid signal. kSigCount is size of an array indexed with signo. So this constant must be 129.

lib/tsan/rtl/tsan_platform.h

57 ↗

(On Diff #17756)

Please move the x86_64 constants on top of __mips64 constants to be consistent with order of archs in the comment above.

Or even better group comment with constants for a single arch:

#if defined(x86_64)
/*
C/C++ on linux/x86_64 and freebsd/x86_64
0000 0000 1000 - 0100 0000 0000: main binary and/or MAP_32BIT mappings
...
*/
const uptr kMetaShadowBeg = 0x300000000000ull;
...
#elif defined(__mips64)
/*
C/C++ on linux/mips64
0100 0000 00 - 0200 0000 00: main binary
...
*/
const uptr kMetaShadowBeg = 0x3000000000ull;
...
#endif

[slthakur]

• Addressed review comments by Dmitry.
• Added changes to enable unit tests.

[slthakur]

This comment was removed by slthakur.

[slthakur]

lib/tsan/tests/CMakeLists.txt

38 ↗

(On Diff #18221)

Hi @samsonov

On X86_64, with the previous patch, standalone compilation was failing with -DCOMPILER_RT_INCLUDE_TESTS=ON because of a cmake error. This was happening because the funtion get_target_flags_for_arch () in add_tsan_unittest macro was revieving incorrect arch value in its first argument.
I have added this change to enable unit tests for all architectures in this patch.

Does this change look good ?

[samsonov]

LGTM. Thank you for working on this!

[dvyukov]

lib/tsan/rtl/tsan_rtl.h

702 ↗

(On Diff #18221)

please change !defined(mips) to defined(x86_64)
people working on aarch64 has a similar change that disables HACKY_CALL for aarch64

[slthakur]

Changed !defined(mips) to defined(x86_64) for HACKY_CALL() in tsan_rtl.h

[slthakur]

• Enabled GotsanRuntimeCheck for x86_64 only.
• Added #ifdef directive to compile HeapEnd() function in C/C++ mode only.

[samsonov]

lib/tsan/CMakeLists.txt

114 ↗

(On Diff #20143)

Move this target definition inside foreach() loop, and put it under if(arch STREQUAL "x86_64")

[slthakur]

Addressed review comments by @samsonov

[slthakur]

Sorry I missed one empty line. Added missing empty line.

[samsonov]

LGTM, feel free to submit after addressing one nit below.

lib/tsan/CMakeLists.txt

111 ↗

(On Diff #20262)

No, I actually meant to put it inside foreach() loop above, where we create a build rule for clang_rt.tsan-${arch}.

[slthakur]

Moved target GotsanRuntimeCheck into foreach() loop as suggested by @samsonov

compiler-rt/trunk/cmake/config-ix.cmake

 filter_available_targets(LSAN_SUPPORTED_ARCH x86_64 mips64 mips64el)
 # LSan common files should be available on all architectures supported
 # by other sanitizers (even if they build into dummy object files).
 filter_available_targets(LSAN_COMMON_SUPPORTED_ARCH
   ${SANITIZER_COMMON_SUPPORTED_ARCH})
 filter_available_targets(MSAN_SUPPORTED_ARCH x86_64 mips64 mips64el)
 filter_available_targets(PROFILE_SUPPORTED_ARCH x86_64 i386 i686 arm mips mips64
   mipsel mips64el aarch64 powerpc64 powerpc64le)
-filter_available_targets(TSAN_SUPPORTED_ARCH x86_64)
+filter_available_targets(TSAN_SUPPORTED_ARCH x86_64 mips64 mips64el)
 filter_available_targets(UBSAN_SUPPORTED_ARCH x86_64 i386 i686 arm aarch64 mips mipsel mips64 mips64el)
 
 if(ANDROID)
   set(OS_NAME "Android")
 else()
   set(OS_NAME "${CMAKE_SYSTEM_NAME}")
 endif()
 

compiler-rt/trunk/lib/tsan/CMakeLists.txt

set(TSAN_HEADERS
   rtl/tsan_sync.h
   rtl/tsan_trace.h
   rtl/tsan_update_shadow_word_inl.h
   rtl/tsan_vector.h)
 
 set(TSAN_RUNTIME_LIBRARIES)
 add_custom_target(tsan)
 # TSan is currently supported on 64-bit Linux only.
-if(CAN_TARGET_x86_64 AND UNIX AND NOT APPLE)
+if(UNIX AND NOT APPLE)
+  foreach(arch ${TSAN_SUPPORTED_ARCH})
+    if(arch STREQUAL "x86_64")
       set(TSAN_ASM_SOURCES rtl/tsan_rtl_amd64.S)
       # Pass ASM file directly to the C++ compiler.
       set_source_files_properties(${TSAN_ASM_SOURCES} PROPERTIES
         LANGUAGE C)
-  set(arch "x86_64")
+      # Sanity check for Go runtime.
+      set(BUILDGO_SCRIPT ${CMAKE_CURRENT_SOURCE_DIR}/go/buildgo.sh)
+      add_custom_target(GotsanRuntimeCheck
+        COMMAND CC=${CMAKE_C_COMPILER} IN_TMPDIR=1 SILENT=1 ${BUILDGO_SCRIPT}
+        DEPENDS clang_rt.tsan-${arch} ${BUILDGO_SCRIPT}
+        WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/go
+        COMMENT "Checking TSan Go runtime..."
+        VERBATIM)
+    else()
+      set(TSAN_ASM_SOURCES)
+    endif()
     add_compiler_rt_runtime(clang_rt.tsan-${arch} ${arch} STATIC
       SOURCES ${TSAN_SOURCES} ${TSAN_ASM_SOURCES}
               $<TARGET_OBJECTS:RTInterception.${arch}>
               $<TARGET_OBJECTS:RTSanitizerCommon.${arch}>
               $<TARGET_OBJECTS:RTSanitizerCommonLibc.${arch}>
       CFLAGS ${TSAN_RTL_CFLAGS}
       DEFS ${TSAN_COMMON_DEFINITIONS})
     list(APPEND TSAN_RUNTIME_LIBRARIES clang_rt.tsan-${arch})
     add_sanitizer_rt_symbols(clang_rt.tsan-${arch} rtl/tsan.syms.extra)
     add_dependencies(tsan clang_rt.tsan-${arch}
       clang_rt.tsan-${arch}-symbols)
+  endforeach()
 endif()
 
 add_dependencies(compiler-rt tsan)
 
-# Sanity check for Go runtime.
-set(BUILDGO_SCRIPT ${CMAKE_CURRENT_SOURCE_DIR}/go/buildgo.sh)
-add_custom_target(GotsanRuntimeCheck
-  COMMAND CC=${CMAKE_C_COMPILER} IN_TMPDIR=1 SILENT=1 ${BUILDGO_SCRIPT}
-  DEPENDS clang_rt.tsan-${arch} ${BUILDGO_SCRIPT}
-  WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/go
-  COMMENT "Checking TSan Go runtime..."
-  VERBATIM)
 
 # Build libcxx instrumented with TSan.
 if(TSAN_SUPPORTED_ARCH AND
    COMPILER_RT_HAS_LIBCXX_SOURCES AND
    COMPILER_RT_TEST_COMPILER_ID STREQUAL "Clang")
   set(LIBCXX_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/libcxx_tsan)
   add_custom_libcxx(libcxx_tsan ${LIBCXX_PREFIX}
     DEPS ${TSAN_RUNTIME_LIBRARIES}
     CFLAGS -fsanitize=thread)
 endif()
 
 if(COMPILER_RT_INCLUDE_TESTS)
   add_subdirectory(tests)
 endif()

compiler-rt/trunk/lib/tsan/rtl/tsan_interceptors.cc

 #define __libc_malloc __malloc
 #define __libc_realloc __realloc
 #define __libc_calloc __calloc
 #define __libc_free __free
 #define stdout __stdoutp
 #define stderr __stderrp
 #endif
 
+#ifdef __mips__
+const int kSigCount = 129;
+#else
 const int kSigCount = 65;
+#endif
 
 struct my_siginfo_t {
   // The size is determined by looking at sizeof of real siginfo_t on linux.
   u64 opaque[128 / sizeof(u64)];
 };
 
+#ifdef __mips__
+struct ucontext_t {
+  u64 opaque[768 / sizeof(u64) + 1];
+};
+#else
 struct ucontext_t {
   // The size is determined by looking at sizeof of real ucontext_t on linux.
   u64 opaque[936 / sizeof(u64) + 1];
 };
+#endif
 
 extern "C" int pthread_attr_init(void *attr);
 extern "C" int pthread_attr_destroy(void *attr);
 DECLARE_REAL(int, pthread_attr_getdetachstate, void *, void *)
 extern "C" int pthread_attr_setstacksize(void *attr, uptr stacksize);
 extern "C" int pthread_key_create(unsigned *key, void (*destructor)(void* v));
 extern "C" int pthread_setspecific(unsigned key, const void *v);
 DECLARE_REAL(int, pthread_mutexattr_gettype, void *, void *)

 const int EOWNERDEAD = 130;
 const int EPOLL_CTL_ADD = 1;
 const int SIGILL = 4;
 const int SIGABRT = 6;
 const int SIGFPE = 8;
 const int SIGSEGV = 11;
 const int SIGPIPE = 13;
 const int SIGTERM = 15;
+#ifdef __mips__
+const int SIGBUS = 10;
+const int SIGSYS = 12;
+#else
 const int SIGBUS = 7;
 const int SIGSYS = 31;
+#endif
 void *const MAP_FAILED = (void*)-1;
 const int PTHREAD_BARRIER_SERIAL_THREAD = -1;
 const int MAP_FIXED = 0x10;
 typedef long long_t;  // NOLINT
 
 // From /usr/include/unistd.h
 # define F_ULOCK 0      /* Unlock a previously locked region.  */
 # define F_LOCK  1      /* Lock a region for exclusive use.  */
 # define F_TLOCK 2      /* Test and lock a region for exclusive use.  */
 # define F_TEST  3      /* Test a region for other processes locks.  */
 
 #define errno (*__errno_location())
 
 typedef void (*sighandler_t)(int sig);
 typedef void (*sigactionhandler_t)(int sig, my_siginfo_t *siginfo, void *uctx);
 
 struct sigaction_t {
+#ifdef __mips__
+  u32 sa_flags;
+#endif
   union {
     sighandler_t sa_handler;
     sigactionhandler_t sa_sigaction;
   };
 #if SANITIZER_FREEBSD
   int sa_flags;
   __sanitizer_sigset_t sa_mask;
 #else
   __sanitizer_sigset_t sa_mask;
+#ifndef __mips__
   int sa_flags;
+#endif
   void (*sa_restorer)();
 #endif
 };
 
 const sighandler_t SIG_DFL = (sighandler_t)0;
 const sighandler_t SIG_IGN = (sighandler_t)1;
 const sighandler_t SIG_ERR = (sighandler_t)-1;
+#ifdef __mips__
+const int SA_SIGINFO = 8;
+const int SIG_SETMASK = 3;
+#else
 const int SA_SIGINFO = 4;
 const int SIG_SETMASK = 2;
+#endif
 
 namespace std {
 struct nothrow_t {};
 }  // namespace std
 
 static sigaction_t sigactions[kSigCount];
 
 namespace __tsan {

compiler-rt/trunk/lib/tsan/rtl/tsan_interface_atomic.cc

 using namespace __tsan;  // NOLINT
 
 // These should match declarations from public tsan_interface_atomic.h header.
 typedef unsigned char      a8;
 typedef unsigned short     a16;  // NOLINT
 typedef unsigned int       a32;
 typedef unsigned long long a64;  // NOLINT
 #if !defined(SANITIZER_GO) && (defined(__SIZEOF_INT128__) \
-    || (__clang_major__ * 100 + __clang_minor__ >= 302))
+    || (__clang_major__ * 100 + __clang_minor__ >= 302)) && !defined(__mips64)
 __extension__ typedef __int128 a128;
 # define __TSAN_HAS_INT128 1
 #else
 # define __TSAN_HAS_INT128 0
 #endif
 
-#ifndef SANITIZER_GO
+#if !defined(SANITIZER_GO) && __TSAN_HAS_INT128
 // Protects emulation of 128-bit atomic operations.
 static StaticSpinMutex mutex128;
 #endif
 
 // Part of ABI, do not change.
 // http://llvm.org/viewvc/llvm-project/libcxx/trunk/include/atomic?view=markup
 typedef enum {
   mo_relaxed,

 template<typename T> T func_cas(volatile T *v, T cmp, T xch) {
   return __sync_val_compare_and_swap(v, cmp, xch);
 }
 
 // clang does not support 128-bit atomic ops.
 // Atomic ops are executed under tsan internal mutex,
 // here we assume that the atomic variables are not accessed
 // from non-instrumented code.
-#if !defined(__GCC_HAVE_SYNC_COMPARE_AND_SWAP_16) && !defined(SANITIZER_GO)
+#if !defined(__GCC_HAVE_SYNC_COMPARE_AND_SWAP_16) && !defined(SANITIZER_GO) \
+    && __TSAN_HAS_INT128
 a128 func_xchg(volatile a128 *v, a128 op) {
   SpinMutexLock lock(&mutex128);
   a128 cmp = *v;
   *v = op;
   return cmp;
 }
 
 a128 func_add(volatile a128 *v, a128 op) {

compiler-rt/trunk/lib/tsan/rtl/tsan_platform.h

 
 #include "tsan_defs.h"
 #include "tsan_trace.h"
 
 namespace __tsan {
 
 #if !defined(SANITIZER_GO)
 
+#if defined(__x86_64__)
 /*
-C/C++ on linux and freebsd
+C/C++ on linux/x86_64 and freebsd/x86_64
 0000 0000 1000 - 0100 0000 0000: main binary and/or MAP_32BIT mappings
 0100 0000 0000 - 0200 0000 0000: -
 0200 0000 0000 - 1000 0000 0000: shadow
 1000 0000 0000 - 3000 0000 0000: -
 3000 0000 0000 - 4000 0000 0000: metainfo (memory blocks and sync objects)
 4000 0000 0000 - 6000 0000 0000: -
 6000 0000 0000 - 6200 0000 0000: traces
 6200 0000 0000 - 7d00 0000 0000: -
 7d00 0000 0000 - 7e00 0000 0000: heap
 7e00 0000 0000 - 7e80 0000 0000: -
 7e80 0000 0000 - 8000 0000 0000: modules and main thread stack
 */
-
 const uptr kMetaShadowBeg = 0x300000000000ull;
 const uptr kMetaShadowEnd = 0x400000000000ull;
 const uptr kTraceMemBeg   = 0x600000000000ull;
 const uptr kTraceMemEnd   = 0x620000000000ull;
 const uptr kShadowBeg     = 0x020000000000ull;
 const uptr kShadowEnd     = 0x100000000000ull;
 const uptr kHeapMemBeg    = 0x7d0000000000ull;
 const uptr kHeapMemEnd    = 0x7e0000000000ull;
 const uptr kLoAppMemBeg   = 0x000000001000ull;
 const uptr kLoAppMemEnd   = 0x010000000000ull;
 const uptr kHiAppMemBeg   = 0x7e8000000000ull;
 const uptr kHiAppMemEnd   = 0x800000000000ull;
 const uptr kAppMemMsk     = 0x7c0000000000ull;
 const uptr kAppMemXor     = 0x020000000000ull;
+const uptr kVdsoBeg       = 0xf000000000000000ull;
+#elif defined(__mips64)
+/*
+C/C++ on linux/mips64
+0100 0000 00 - 0200 0000 00: main binary
+0200 0000 00 - 1400 0000 00: -
[phone] - 2400 0000 00: shadow
[phone] - 3000 0000 00: -
[phone] - 4000 0000 00: metainfo (memory blocks and sync objects)
[phone] - 6000 0000 00: -
[phone] - 6200 0000 00: traces
[phone] - fe00 0000 00: -
+fe00 0000 00 - ff00 0000 00: heap
+ff00 0000 00 - ff80 0000 00: -
+ff80 0000 00 - ffff ffff ff: modules and main thread stack
+*/
+const uptr kMetaShadowBeg = 0x3000000000ull;
+const uptr kMetaShadowEnd = 0x4000000000ull;
+const uptr kTraceMemBeg   = 0x6000000000ull;
+const uptr kTraceMemEnd   = 0x6200000000ull;
+const uptr kShadowBeg     = 0x1400000000ull;
+const uptr kShadowEnd     = 0x2400000000ull;
+const uptr kHeapMemBeg    = 0xfe00000000ull;
+const uptr kHeapMemEnd    = 0xff00000000ull;
+const uptr kLoAppMemBeg   = 0x0100000000ull;
+const uptr kLoAppMemEnd   = 0x0200000000ull;
+const uptr kHiAppMemBeg   = 0xff80000000ull;
+const uptr kHiAppMemEnd   = 0xffffffffffull;
+const uptr kAppMemMsk     = 0xfc00000000ull;
+const uptr kAppMemXor     = 0x0400000000ull;
+const uptr kVdsoBeg       = 0xfffff00000ull;
+#endif
 
 ALWAYS_INLINE
 bool IsAppMem(uptr mem) {
   return (mem >= kHeapMemBeg && mem < kHeapMemEnd) ||
          (mem >= kLoAppMemBeg && mem < kLoAppMemEnd) ||
          (mem >= kHiAppMemBeg && mem < kHiAppMemEnd);
 }
 

compiler-rt/trunk/lib/tsan/rtl/tsan_platform_linux.cc

 void *__libc_stack_end = 0;
 #endif
 
 namespace __tsan {
 
 static uptr g_data_start;
 static uptr g_data_end;
 
-const uptr kPageSize = 4096;
-
 enum {
   MemTotal  = 0,
   MemShadow = 1,
   MemMeta   = 2,
   MemFile   = 3,
   MemMmap   = 4,
   MemTrace  = 5,
   MemHeap   = 6,
#endif
   // Fill the file with kShadowRodata.
   const uptr kMarkerSize = 512 * 1024 / sizeof(u64);
   InternalScopedBuffer<u64> marker(kMarkerSize);
   // volatile to prevent insertion of memset
   for (volatile u64 *p = marker.data(); p < marker.data() + kMarkerSize; p++)
     *p = kShadowRodata;
   internal_write(fd, marker.data(), marker.size());
   // Map the file into memory.
-  uptr page = internal_mmap(0, kPageSize, PROT_READ | PROT_WRITE,
+  uptr page = internal_mmap(0, GetPageSizeCached(), PROT_READ | PROT_WRITE,
                             MAP_PRIVATE | MAP_ANONYMOUS, fd, 0);
   if (internal_iserror(page)) {
     internal_close(fd);
     return;
   }
   // Map the file into shadow of .rodata sections.
   MemoryMappingLayout proc_maps(/*cache_enabled*/true);
   uptr start, end, offset, prot;
Printf("FATAL: Make sure to compile with -fPIE and "
                "to link with -pie (%p, %p).\n", shadow, kShadowBeg);
     Die();
   }
   // This memory range is used for thread stacks and large user mmaps.
   // Frequently a thread uses only a small part of stack and similarly
   // a program uses a small part of large mmap. On some programs
   // we see 20% memory usage reduction without huge pages for this range.
   // FIXME: don't use constants here.
-  NoHugePagesInRegion(MemToShadow(0x7f0000000000ULL),
-                      0x10000000000ULL * kShadowMultiplier);
+#if defined(__x86_64__)
+  const uptr kMadviseRangeBeg  = 0x7f0000000000ull;
+  const uptr kMadviseRangeSize = 0x010000000000ull;
+#elif defined(__mips64)
+  const uptr kMadviseRangeBeg  = 0xff00000000ull;
+  const uptr kMadviseRangeSize = 0x0100000000ull;
+#endif
+  NoHugePagesInRegion(MemToShadow(kMadviseRangeBeg),
+                      kMadviseRangeSize * kShadowMultiplier);
   if (common_flags()->use_madv_dontdump)
     DontDumpShadowMemory(kShadowBeg, kShadowEnd - kShadowBeg);
   DPrintf("memory shadow: %zx-%zx (%zuGB)\n",
       kShadowBeg, kShadowEnd,
       (kShadowEnd - kShadowBeg) >> 30);
 
   // Map meta shadow.
   uptr meta_size = kMetaShadowEnd - kMetaShadowBeg;

 static void CheckAndProtect() {
   // Ensure that the binary is indeed compiled with -pie.
   MemoryMappingLayout proc_maps(true);
   uptr p, end;
   while (proc_maps.Next(&p, &end, 0, 0, 0, 0)) {
     if (IsAppMem(p))
       continue;
     if (p >= kHeapMemEnd &&
-        p < kHeapMemEnd + PrimaryAllocator::AdditionalSize())
+        p < HeapEnd())
       continue;
-    if (p >= 0xf000000000000000ull)  // vdso
+    if (p >= kVdsoBeg)  // vdso
       break;
     Printf("FATAL: ThreadSanitizer: unexpected memory mapping %p-%p\n", p, end);
     Die();
   }
 
   ProtectRange(kLoAppMemEnd, kShadowBeg);
   ProtectRange(kShadowEnd, kMetaShadowBeg);
   ProtectRange(kMetaShadowEnd, kTraceMemBeg);
   // Memory for traces is mapped lazily in MapThreadTrace.
   // Protect the whole range for now, so that user does not map something here.
   ProtectRange(kTraceMemBeg, kTraceMemEnd);
   ProtectRange(kTraceMemEnd, kHeapMemBeg);
-  ProtectRange(kHeapMemEnd + PrimaryAllocator::AdditionalSize(), kHiAppMemBeg);
+  ProtectRange(HeapEnd(), kHiAppMemBeg);
 }
 #endif  // #ifndef SANITIZER_GO
 
 void InitializePlatform() {
   DisableCoreDumperIfNecessary();
 
   // Go maps shadow memory lazily and works fine with limited address space.
   // Unlimited stack is not a problem as well, because the executable

compiler-rt/trunk/lib/tsan/rtl/tsan_rtl.h

 #if SANITIZER_WORDSIZE != 64
 # error "ThreadSanitizer is supported only on 64-bit platforms"
 #endif
 
 namespace __tsan {
 
 #ifndef SANITIZER_GO
 struct MapUnmapCallback;
+#ifdef __mips64
+static const uptr kAllocatorSpace = 0;
+static const uptr kAllocatorSize = SANITIZER_MMAP_RANGE_SIZE;
+static const uptr kAllocatorRegionSizeLog = 20;
+static const uptr kAllocatorNumRegions =
+    kAllocatorSize >> kAllocatorRegionSizeLog;
+typedef TwoLevelByteMap<(kAllocatorNumRegions >> 12), 1 << 12,
+    MapUnmapCallback> ByteMap;
+typedef SizeClassAllocator32<kAllocatorSpace, kAllocatorSize, 0,
+    CompactSizeClassMap, kAllocatorRegionSizeLog, ByteMap,
+    MapUnmapCallback> PrimaryAllocator;
+#else
 typedef SizeClassAllocator64<kHeapMemBeg, kHeapMemEnd - kHeapMemBeg, 0,
     DefaultSizeClassMap, MapUnmapCallback> PrimaryAllocator;
+#endif
 typedef SizeClassAllocatorLocalCache<PrimaryAllocator> AllocatorCache;
 typedef LargeMmapAllocator<MapUnmapCallback> SecondaryAllocator;
 typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,
     SecondaryAllocator> Allocator;
 Allocator *allocator();
 #endif
 
 void TsanCheckFailed(const char *file, int line, const char *cond,

 #endif
   }
   Event *trace = (Event*)GetThreadTrace(fs.tid());
   Event *evp = &trace[pos];
   Event ev = (u64)addr | ((u64)typ << 61);
   *evp = ev;
 }
 
+#ifndef SANITIZER_GO
+uptr ALWAYS_INLINE HeapEnd() {
+#if SANITIZER_CAN_USE_ALLOCATOR64
+  return kHeapMemEnd + PrimaryAllocator::AdditionalSize();
+#else
+  return kHeapMemEnd;
+#endif
+}
+#endif
+
 }  // namespace __tsan
 
 #endif  // TSAN_RTL_H

compiler-rt/trunk/lib/tsan/rtl/tsan_rtl.cc

for (int i = 0;
     }
   }
 }
 
 static void StartBackgroundThread() {
   ctx->background_thread = internal_start_thread(&BackgroundThread, 0);
 }
 
+#ifndef __mips__
 static void StopBackgroundThread() {
   atomic_store(&ctx->stop_background_thread, 1, memory_order_relaxed);
   internal_join_thread(ctx->background_thread);
   ctx->background_thread = 0;
 }
 #endif
+#endif
 
 void DontNeedShadowFor(uptr addr, uptr size) {
   uptr shadow_beg = MemToShadow(addr);
   uptr shadow_end = MemToShadow(addr + size);
   FlushUnneededShadowMemory(shadow_beg, shadow_end - shadow_beg);
 }
 
 void MapShadow(uptr addr, uptr size) {
#ifndef SANITIZER_GO
   InitializeShadowMemory();
 #endif
   // Setup correct file descriptor for error reports.
   __sanitizer_set_report_path(common_flags()->log_path);
   InitializeSuppressions();
 #ifndef SANITIZER_GO
   InitializeLibIgnore();
   Symbolizer::GetOrInit()->AddHooks(EnterSymbolizer, ExitSymbolizer);
+  // On MIPS, TSan initialization is run before
+  // __pthread_initialize_minimal_internal() is finished, so we can not spawn
+  // new threads.
+#ifndef __mips__
   StartBackgroundThread();
   SetSandboxingCallback(StopBackgroundThread);
 #endif
+#endif
   if (common_flags()->detect_deadlocks)
     ctx->dd = DDetector::Create(flags());
 
   VPrintf(1, "***** Running under ThreadSanitizer v2 (pid %d) *****\n",
           (int)internal_getpid());
 
   // Initialize thread 0.
   int tid = ThreadCreate(thr, 0, 0, true);
if (kGoMode || size < common_flags()->clear_shadow_mmap_threshold) {
     // FIXME: may overwrite a part outside the region
     for (uptr i = 0; i < size / kShadowCell * kShadowCnt;) {
       p[i++] = val;
       for (uptr j = 1; j < kShadowCnt; j++)
         p[i++] = 0;
     }
   } else {
     // The region is big, reset only beginning and end.
-    const uptr kPageSize = 4096;
+    const uptr kPageSize = GetPageSizeCached();
     u64 *begin = (u64*)MemToShadow(addr);
     u64 *end = begin + size / kShadowCell * kShadowCnt;
     u64 *p = begin;
     // Set at least first kPageSize/2 to page boundary.
     while ((p < begin + kPageSize / kShadowSize / 2) || ((uptr)p % kPageSize)) {
       *p++ = val;
       for (uptr j = 1; j < kShadowCnt; j++)
         *p++ = 0;

compiler-rt/trunk/lib/tsan/tests/CMakeLists.txt

macro(tsan_compile obj_list source arch)
   clang_compile(${output_obj} ${source}
           CFLAGS ${TSAN_UNITTEST_CFLAGS} ${TARGET_CFLAGS}
           DEPS ${COMPILE_DEPS})
   list(APPEND ${obj_list} ${output_obj})
 endmacro()
 
 macro(add_tsan_unittest testname)
   # Build unit tests only for 64-bit Linux.
-  if(UNIX AND NOT APPLE AND CAN_TARGET_x86_64)
+  if(UNIX AND NOT APPLE)
+    foreach(arch ${TSAN_SUPPORTED_ARCH})
       parse_arguments(TEST "SOURCES;HEADERS" "" ${ARGN})
       set(TEST_OBJECTS)
       foreach(SOURCE ${TEST_SOURCES} ${COMPILER_RT_GTEST_SOURCE})
-      tsan_compile(TEST_OBJECTS ${SOURCE} x86_64 ${TEST_HEADERS})
+        tsan_compile(TEST_OBJECTS ${SOURCE} ${arch} ${TEST_HEADERS})
       endforeach()
       get_target_flags_for_arch(${arch} TARGET_LINK_FLAGS)
       set(TEST_DEPS ${TEST_OBJECTS})
       if(NOT COMPILER_RT_STANDALONE_BUILD)
         list(APPEND TEST_DEPS tsan)
       endif()
       # FIXME: Looks like we should link TSan with just-built runtime,
       # and not rely on -fsanitize=thread, as these tests are essentially
       # unit tests.
       add_compiler_rt_test(TsanUnitTests ${testname}
               OBJECTS ${TEST_OBJECTS}
               DEPS ${TEST_DEPS}
               LINK_FLAGS ${TARGET_LINK_FLAGS}
                          -fsanitize=thread
                          -lstdc++ -lm)
+    endforeach()
   endif()
 endmacro()
 
 if(COMPILER_RT_CAN_EXECUTE_TESTS)
   add_subdirectory(rtl)
   add_subdirectory(unit)
 endif()