[Sanitizers] TSan allocator set errno on failure.

[alekseyshl]

Summary

Set proper errno code on allocation failures and change realloc, pvalloc,
aligned_alloc, memalign and posix_memalign implementation to satisfy
their man-specified requirements.

Modify allocator API implementation to bring it closer to other
sanitizers allocators.

[dvyukov]

lib/tsan/rtl/tsan_mman.cc

174

Please don't prefix functions, variables and types in tsan with tsan. It's all tsan already. It's just plain pointless to prefix everything in tsan with tsan, everything in asan with asan, etc, and does not add any information for reader about purpose of a function or a type.

Also, diff will be much smaller if you name this function user_alloc and rename the old one to, say, user_alloc_noerrno. It will be just few lines instead of churning the whole codebase.

[alekseyshl]

• Do not use "tsan_" prefix.

[alekseyshl]

lib/tsan/rtl/tsan_mman.cc

174

Well, not sure about a few lines, but will do. It just seem weird to mention errno in the code which does not care about errno at all. Maybe user_alloc_internal?

[dvyukov]

lib/tsan/rtl/tsan_mman.cc

174

Much better now. Thanks.
user_alloc_internal is fine.

lib/tsan/tests/unit/tsan_mman_test.cc

70–72

Is there a particular reason for this change? If not, then I would prefer to leave the current behavior. It worked for 5 years. We don't know what will happen with this one.

test/tsan/allocator_returns_null.cc

40

Are C++ tsan tests run on windows? That's surprising to me.

[dvyukov]

lib/tsan/tests/unit/tsan_mman_test.cc

70–72

Note: the current behavior is correct according to both POSIX, linux man pages and C:

If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned.

If size was equal to 0, either NULL or a pointer suitable to be passed to free() is returned.

[alekseyshl]

lib/tsan/tests/unit/tsan_mman_test.cc

70–72

Wait, that's what the man I quoted in comments: "if size is equal to zero, and ptr is not NULL, then the call is equivalent to free(ptr)", so I'm freeing the pointer and it seems only natural to return NULL in this case, cause what else would I return?

What you're saying is correct for user_realloc(thr, pc, NULL, 0) case.

[alekseyshl]

• Address comments

[alekseyshl]

lib/tsan/tests/unit/tsan_mman_test.cc

70–72

What I mean to say, it seems strange to return a dangling pointer and stdlib's realloc returns NULL in this case anyway.

user_realloc(thr, pc, NULL, 0) is tested first hand in this test.

test/tsan/allocator_returns_null.cc

40

Yep, got carried away unifying tests with other sanitizers.

[dvyukov]

lib/tsan/tests/unit/tsan_mman_test.cc

70–72

Okay, let's try this.

lib/tsan/rtl/tsan_interceptors.cc

 
 #if !SANITIZER_MAC
 TSAN_INTERCEPTOR(void*, malloc, uptr size) {
   if (cur_thread()->in_symbolizer)
     return InternalAlloc(size);
   void *p = 0;
   {
     SCOPED_INTERCEPTOR_RAW(malloc, size);
-    p = user_alloc(thr, pc, size);
+    p = tsan_malloc(thr, pc, size);
   }
   invoke_malloc_hook(p, size);
   return p;
 }
 
 TSAN_INTERCEPTOR(void*, __libc_memalign, uptr align, uptr sz) {
   SCOPED_TSAN_INTERCEPTOR(__libc_memalign, align, sz);
-  return user_alloc(thr, pc, sz, align);
+  return tsan_memalign(thr, pc, align, sz);
 }
 
 TSAN_INTERCEPTOR(void*, calloc, uptr size, uptr n) {
   if (cur_thread()->in_symbolizer)
     return InternalCalloc(size, n);
   void *p = 0;
   {
     SCOPED_INTERCEPTOR_RAW(calloc, size, n);
-    p = user_calloc(thr, pc, size, n);
+    p = tsan_calloc(thr, pc, size, n);
   }
   invoke_malloc_hook(p, n * size);
   return p;
 }
 
 TSAN_INTERCEPTOR(void*, realloc, void *p, uptr size) {
   if (cur_thread()->in_symbolizer)
     return InternalRealloc(p, size);
   if (p)
     invoke_free_hook(p);
   {
     SCOPED_INTERCEPTOR_RAW(realloc, p, size);
-    p = user_realloc(thr, pc, p, size);
+    p = tsan_realloc(thr, pc, p, size);
   }
   invoke_malloc_hook(p, size);
   return p;
 }
 
 TSAN_INTERCEPTOR(void, free, void *p) {
   if (p == 0)
     return;
   if (cur_thread()->in_symbolizer)
     return InternalFree(p);
   invoke_free_hook(p);
   SCOPED_INTERCEPTOR_RAW(free, p);
-  user_free(thr, pc, p);
+  tsan_free(thr, pc, p);
 }
 
 TSAN_INTERCEPTOR(void, cfree, void *p) {
   if (p == 0)
     return;
   if (cur_thread()->in_symbolizer)
     return InternalFree(p);
   invoke_free_hook(p);
   SCOPED_INTERCEPTOR_RAW(cfree, p);
-  user_free(thr, pc, p);
+  tsan_free(thr, pc, p);
 }
 
 TSAN_INTERCEPTOR(uptr, malloc_usable_size, void *p) {
   SCOPED_INTERCEPTOR_RAW(malloc_usable_size, p);
-  return user_alloc_usable_size(p);
+  return tsan_alloc_usable_size(p);
 }
 #endif
 
 TSAN_INTERCEPTOR(char*, strcpy, char *dst, const char *src) {  // NOLINT
   SCOPED_TSAN_INTERCEPTOR(strcpy, dst, src);  // NOLINT
   uptr srclen = internal_strlen(src);
   MemoryAccessRange(thr, pc, (uptr)dst, srclen + 1, true);
   MemoryAccessRange(thr, pc, (uptr)src, srclen + 1, false);
TSAN_INTERCEPTOR(int, munmap, void *addr, long_t sz) {
   }
   int res = REAL(munmap)(addr, sz);
   return res;
 }
 
 #if SANITIZER_LINUX
 TSAN_INTERCEPTOR(void*, memalign, uptr align, uptr sz) {
   SCOPED_INTERCEPTOR_RAW(memalign, align, sz);
-  return user_alloc(thr, pc, sz, align);
+  return tsan_memalign(thr, pc, align, sz);
 }
 #define TSAN_MAYBE_INTERCEPT_MEMALIGN TSAN_INTERCEPT(memalign)
 #else
 #define TSAN_MAYBE_INTERCEPT_MEMALIGN
 #endif
 
 #if !SANITIZER_MAC
 TSAN_INTERCEPTOR(void*, aligned_alloc, uptr align, uptr sz) {
-  SCOPED_INTERCEPTOR_RAW(memalign, align, sz);
-  return user_alloc(thr, pc, sz, align);
+  SCOPED_INTERCEPTOR_RAW(aligned_alloc, align, sz);
+  return tsan_aligned_alloc(thr, pc, align, sz);
 }
 
 TSAN_INTERCEPTOR(void*, valloc, uptr sz) {
   SCOPED_INTERCEPTOR_RAW(valloc, sz);
-  return user_alloc(thr, pc, sz, GetPageSizeCached());
+  return tsan_valloc(thr, pc, sz);
 }
 #endif
 
 #if SANITIZER_LINUX
 TSAN_INTERCEPTOR(void*, pvalloc, uptr sz) {
   SCOPED_INTERCEPTOR_RAW(pvalloc, sz);
-  sz = RoundUp(sz, GetPageSizeCached());
-  return user_alloc(thr, pc, sz, GetPageSizeCached());
+  return tsan_pvalloc(thr, pc, sz);
 }
 #define TSAN_MAYBE_INTERCEPT_PVALLOC TSAN_INTERCEPT(pvalloc)
 #else
 #define TSAN_MAYBE_INTERCEPT_PVALLOC
 #endif
 
 #if !SANITIZER_MAC
 TSAN_INTERCEPTOR(int, posix_memalign, void **memptr, uptr align, uptr sz) {
   SCOPED_INTERCEPTOR_RAW(posix_memalign, memptr, align, sz);
-  *memptr = user_alloc(thr, pc, sz, align);
-  return 0;
+  return tsan_posix_memalign(thr, pc, memptr, align, sz);
 }
 #endif
 
 // __cxa_guard_acquire and friends need to be intercepted in a special way -
 // regular interceptors will break statically-linked libstdc++. Linux
 // interceptors are especially defined as weak functions (so that they don't
 // cause link errors when user defines them as well). So they silently
 // auto-disable themselves when such symbol is already present in the binary. If

lib/tsan/rtl/tsan_malloc_mac.cc

 using namespace __tsan;
 #define COMMON_MALLOC_ZONE_NAME "tsan"
 #define COMMON_MALLOC_ENTER()
 #define COMMON_MALLOC_SANITIZER_INITIALIZED (cur_thread()->is_inited)
 #define COMMON_MALLOC_FORCE_LOCK()
 #define COMMON_MALLOC_FORCE_UNLOCK()
 #define COMMON_MALLOC_MEMALIGN(alignment, size) \
   void *p =                                     \
-      user_alloc(cur_thread(), StackTrace::GetCurrentPc(), size, alignment)
+      tsan_memalign(cur_thread(), StackTrace::GetCurrentPc(), alignment, size)
 #define COMMON_MALLOC_MALLOC(size)                             \
   if (cur_thread()->in_symbolizer) return InternalAlloc(size); \
   SCOPED_INTERCEPTOR_RAW(malloc, size);                        \
-  void *p = user_alloc(thr, pc, size)
+  void *p = tsan_malloc(thr, pc, size)
 #define COMMON_MALLOC_REALLOC(ptr, size)                              \
   if (cur_thread()->in_symbolizer) return InternalRealloc(ptr, size); \
   SCOPED_INTERCEPTOR_RAW(realloc, ptr, size);                         \
-  void *p = user_realloc(thr, pc, ptr, size)
+  void *p = tsan_realloc(thr, pc, ptr, size)
 #define COMMON_MALLOC_CALLOC(count, size)                              \
   if (cur_thread()->in_symbolizer) return InternalCalloc(count, size); \
   SCOPED_INTERCEPTOR_RAW(calloc, size, count);                         \
-  void *p = user_calloc(thr, pc, size, count)
+  void *p = tsan_calloc(thr, pc, size, count)
 #define COMMON_MALLOC_VALLOC(size)                            \
   if (cur_thread()->in_symbolizer)                            \
     return InternalAlloc(size, nullptr, GetPageSizeCached()); \
   SCOPED_INTERCEPTOR_RAW(valloc, size);                       \
-  void *p = user_alloc(thr, pc, size, GetPageSizeCached())
+  void *p = tsan_valloc(thr, pc, size)
 #define COMMON_MALLOC_FREE(ptr)                              \
   if (cur_thread()->in_symbolizer) return InternalFree(ptr); \
   SCOPED_INTERCEPTOR_RAW(free, ptr);                         \
-  user_free(thr, pc, ptr)
-#define COMMON_MALLOC_SIZE(ptr) uptr size = user_alloc_usable_size(ptr);
+  tsan_free(thr, pc, ptr)
+#define COMMON_MALLOC_SIZE(ptr) uptr size = tsan_alloc_usable_size(ptr);
 #define COMMON_MALLOC_FILL_STATS(zone, stats)
 #define COMMON_MALLOC_REPORT_UNKNOWN_REALLOC(ptr, zone_ptr, zone_name) \
   (void)zone_name; \
   Report("mz_realloc(%p) -- attempting to realloc unallocated memory.\n", ptr);
 #define COMMON_MALLOC_NAMESPACE __tsan
 
 #include "sanitizer_common/sanitizer_malloc_mac.inc"
 
 #endif

lib/tsan/rtl/tsan_mman.h

 void ReplaceSystemMalloc();
 void AllocatorProcStart(Processor *proc);
 void AllocatorProcFinish(Processor *proc);
 void AllocatorPrintStats();
 
 // For user allocations.
 void *user_alloc(ThreadState *thr, uptr pc, uptr sz,
                  uptr align = kDefaultAlignment, bool signal = true);
-void *user_calloc(ThreadState *thr, uptr pc, uptr sz, uptr n);
 // Does not accept NULL.
 void user_free(ThreadState *thr, uptr pc, void *p, bool signal = true);
-void *user_realloc(ThreadState *thr, uptr pc, void *p, uptr sz);
-void *user_alloc_aligned(ThreadState *thr, uptr pc, uptr sz, uptr align);
-uptr user_alloc_usable_size(const void *p);
+// Interceptor implementations.
+void *tsan_malloc(ThreadState *thr, uptr pc, uptr sz);
+inline void tsan_free(ThreadState *thr, uptr pc, void *p) {
+  user_free(thr, pc, p);
+}
+void *tsan_calloc(ThreadState *thr, uptr pc, uptr sz, uptr n);
+void *tsan_realloc(ThreadState *thr, uptr pc, void *p, uptr sz);
+void *tsan_memalign(ThreadState *thr, uptr pc, uptr align, uptr sz);
+int tsan_posix_memalign(ThreadState *thr, uptr pc, void **memptr, uptr align,
+                        uptr sz);
+void *tsan_aligned_alloc(ThreadState *thr, uptr pc, uptr align, uptr sz);
+void *tsan_valloc(ThreadState *thr, uptr pc, uptr sz);
+void *tsan_pvalloc(ThreadState *thr, uptr pc, uptr sz);
+uptr tsan_alloc_usable_size(const void *p);
 
 // Invoking malloc/free hooks that may be installed by the user.
 void invoke_malloc_hook(void *ptr, uptr size);
 void invoke_free_hook(void *ptr);
 
 enum MBlockType {
   MBlockScopedBuf,
   MBlockString,

lib/tsan/rtl/tsan_mman.cc

static void SignalUnsafeCall(ThreadState *thr, uptr pc) {
   rep.AddStack(stack, true);
   OutputReport(thr, rep);
 }
 
 void *user_alloc(ThreadState *thr, uptr pc, uptr sz, uptr align, bool signal) {
   if ((sz >= (1ull << 40)) || (align >= (1ull << 40)))
     return Allocator::FailureHandler::OnBadRequest();
   void *p = allocator()->Allocate(&thr->proc()->alloc_cache, sz, align);
-  if (p == 0)
+  if (UNLIKELY(p == 0))
     return 0;
   if (ctx && ctx->initialized)
     OnUserAlloc(thr, pc, (uptr)p, sz, true);
   if (signal)
     SignalUnsafeCall(thr, pc);
   return p;
 }
 
-void *user_calloc(ThreadState *thr, uptr pc, uptr size, uptr n) {
-  if (CheckForCallocOverflow(size, n))
-    return Allocator::FailureHandler::OnBadRequest();
-  void *p = user_alloc(thr, pc, n * size);
-  if (p)
-    internal_memset(p, 0, n * size);
-  return p;
-}
-
 void user_free(ThreadState *thr, uptr pc, void *p, bool signal) {
   ScopedGlobalProcessor sgp;
   if (ctx && ctx->initialized)
     OnUserFree(thr, pc, (uptr)p, true);
   allocator()->Deallocate(&thr->proc()->alloc_cache, p);
   if (signal)
     SignalUnsafeCall(thr, pc);
 }
 
+void *tsan_malloc(ThreadState *thr, uptr pc, uptr sz) {
+  return SetErrnoOnNull(user_alloc(thr, pc, sz, kDefaultAlignment));
+}
+
+void *tsan_calloc(ThreadState *thr, uptr pc, uptr size, uptr n) {
+  if (UNLIKELY(CheckForCallocOverflow(size, n)))
+    return SetErrnoOnNull(Allocator::FailureHandler::OnBadRequest());
+  void *p = user_alloc(thr, pc, n * size);
+  if (p)
+    internal_memset(p, 0, n * size);
+  return SetErrnoOnNull(p);
+}
+
 void OnUserAlloc(ThreadState *thr, uptr pc, uptr p, uptr sz, bool write) {
   DPrintf("#%d: alloc(%zu) = %p\n", thr->tid, sz, p);
   ctx->metamap.AllocBlock(thr, pc, p, sz);
   if (write && thr->ignore_reads_and_writes == 0)
     MemoryRangeImitateWrite(thr, pc, (uptr)p, sz);
   else
     MemoryResetRange(thr, pc, (uptr)p, sz);
 }
 
 void OnUserFree(ThreadState *thr, uptr pc, uptr p, bool write) {
   CHECK_NE(p, (void*)0);
   uptr sz = ctx->metamap.FreeBlock(thr->proc(), p);
   DPrintf("#%d: free(%p, %zu)\n", thr->tid, p, sz);
   if (write && thr->ignore_reads_and_writes == 0)
     MemoryRangeFreed(thr, pc, (uptr)p, sz);
 }
 
-void *user_realloc(ThreadState *thr, uptr pc, void *p, uptr sz) {
+void *tsan_realloc(ThreadState *thr, uptr pc, void *p, uptr sz) {
   // FIXME: Handle "shrinking" more efficiently,
   // it seems that some software actually does this.
-  void *p2 = user_alloc(thr, pc, sz);
-  if (p2 == 0)
-    return 0;
-  if (p) {
-    uptr oldsz = user_alloc_usable_size(p);
-    internal_memcpy(p2, p, min(oldsz, sz));
+  if (!p)
+    return SetErrnoOnNull(user_alloc(thr, pc, sz));
+  if (!sz) {
     user_free(thr, pc, p);
+    return nullptr;
   }
-  return p2;
+  void *new_p = user_alloc(thr, pc, sz);
+  if (new_p) {
+    uptr old_sz = tsan_alloc_usable_size(p);
+    internal_memcpy(new_p, p, min(old_sz, sz));
+    user_free(thr, pc, p);
+  }
+  return SetErrnoOnNull(new_p);
+}
+
+void *tsan_memalign(ThreadState *thr, uptr pc, uptr align, uptr sz) {
+  if (UNLIKELY(!IsPowerOfTwo(align))) {
+    errno = errno_EINVAL;
+    return Allocator::FailureHandler::OnBadRequest();
+  }
+  return SetErrnoOnNull(user_alloc(thr, pc, sz, align));
+}
+
+int tsan_posix_memalign(ThreadState *thr, uptr pc, void **memptr, uptr align,
+                        uptr sz) {
+  if (UNLIKELY(!CheckPosixMemalignAlignment(align))) {
+    Allocator::FailureHandler::OnBadRequest();
+    return errno_EINVAL;
+  }
+  void *ptr = user_alloc(thr, pc, sz, align);
+  if (UNLIKELY(!ptr))
+    return errno_ENOMEM;
+  CHECK(IsAligned((uptr)ptr, align));
+  *memptr = ptr;
+  return 0;
+}
+
+void *tsan_aligned_alloc(ThreadState *thr, uptr pc, uptr align, uptr sz) {
+  if (UNLIKELY(!CheckAlignedAllocAlignmentAndSize(align, sz))) {
+    errno = errno_EINVAL;
+    return Allocator::FailureHandler::OnBadRequest();
+  }
+  return SetErrnoOnNull(user_alloc(thr, pc, sz, align));
+}
+
+void *tsan_valloc(ThreadState *thr, uptr pc, uptr sz) {
+  return SetErrnoOnNull(user_alloc(thr, pc, sz, GetPageSizeCached()));
+}
+
+void *tsan_pvalloc(ThreadState *thr, uptr pc, uptr sz) {
+  uptr PageSize = GetPageSizeCached();
+  // pvalloc(0) should allocate one page.
+  sz = sz ? RoundUpTo(sz, PageSize) : PageSize;
+  return SetErrnoOnNull(user_alloc(thr, pc, sz, PageSize));
 }
 
-uptr user_alloc_usable_size(const void *p) {
+uptr tsan_alloc_usable_size(const void *p) {
   if (p == 0)
     return 0;
   MBlock *b = ctx->metamap.GetBlock((uptr)p);
   if (!b)
     return 0;  // Not a valid pointer.
   if (b->siz == 0)
     return 1;  // Zero-sized allocations are actually 1 byte.
   return b->siz;
uptr __sanitizer_get_estimated_allocated_size(uptr size) {
   return size;
 }
 
 int __sanitizer_get_ownership(const void *p) {
   return allocator()->GetBlockBegin(p) != 0;
 }
 
 uptr __sanitizer_get_allocated_size(const void *p) {
-  return user_alloc_usable_size(p);
+  return tsan_alloc_usable_size(p);
 }
 
 void __tsan_on_thread_idle() {
   ThreadState *thr = cur_thread();
   thr->clock.ResetCached(&thr->proc()->clock_cache);
   thr->last_sleep_clock.ResetCached(&thr->proc()->clock_cache);
   allocator()->SwallowCache(&thr->proc()->alloc_cache);
   internal_allocator()->SwallowCache(&thr->proc()->internal_alloc_cache);
   ctx->metamap.OnProcIdle(thr->proc());
 }
 }  // extern "C"

lib/tsan/rtl/tsan_new_delete.cc

 
 // TODO(alekseys): throw std::bad_alloc instead of dying on OOM.
 #define OPERATOR_NEW_BODY(mangled_name, nothrow) \
   if (cur_thread()->in_symbolizer) \
     return InternalAlloc(size); \
   void *p = 0; \
   {  \
     SCOPED_INTERCEPTOR_RAW(mangled_name, size); \
-    p = user_alloc(thr, pc, size); \
+    p = tsan_malloc(thr, pc, size); \
     if (!nothrow && UNLIKELY(!p)) DieOnFailure::OnOOM(); \
   }  \
   invoke_malloc_hook(p, size);  \
   return p;
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void *operator new(__sanitizer::uptr size);
 void *operator new(__sanitizer::uptr size) {

 }
 
 #define OPERATOR_DELETE_BODY(mangled_name) \
   if (ptr == 0) return;  \
   if (cur_thread()->in_symbolizer) \
     return InternalFree(ptr); \
   invoke_free_hook(ptr);  \
   SCOPED_INTERCEPTOR_RAW(mangled_name, ptr);  \
-  user_free(thr, pc, ptr);
+  tsan_free(thr, pc, ptr);
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void operator delete(void *ptr) NOEXCEPT;
 void operator delete(void *ptr) NOEXCEPT {
   OPERATOR_DELETE_BODY(_ZdlPv);
 }
 
 SANITIZER_INTERFACE_ATTRIBUTE

lib/tsan/tests/unit/tsan_mman_test.cc

TEST(Mman, Internal) {
   }
   internal_free(p);
   internal_free(p2);
 }
 
 TEST(Mman, User) {
   ThreadState *thr = cur_thread();
   uptr pc = 0;
-  char *p = (char*)user_alloc(thr, pc, 10);
+  char *p = (char*)tsan_malloc(thr, pc, 10);
   EXPECT_NE(p, (char*)0);
-  char *p2 = (char*)user_alloc(thr, pc, 20);
+  char *p2 = (char*)tsan_malloc(thr, pc, 20);
   EXPECT_NE(p2, (char*)0);
   EXPECT_NE(p2, p);
-  EXPECT_EQ(10U, user_alloc_usable_size(p));
-  EXPECT_EQ(20U, user_alloc_usable_size(p2));
-  user_free(thr, pc, p);
-  user_free(thr, pc, p2);
+  EXPECT_EQ(10U, tsan_alloc_usable_size(p));
+  EXPECT_EQ(20U, tsan_alloc_usable_size(p2));
+  tsan_free(thr, pc, p);
+  tsan_free(thr, pc, p2);
 }
 
 TEST(Mman, UserRealloc) {
   ThreadState *thr = cur_thread();
   uptr pc = 0;
   {
-    void *p = user_realloc(thr, pc, 0, 0);
+    void *p = tsan_realloc(thr, pc, 0, 0);
     // Realloc(NULL, N) is equivalent to malloc(N), thus must return
     // non-NULL pointer.
     EXPECT_NE(p, (void*)0);
   }
   {
-    void *p = user_realloc(thr, pc, 0, 100);
+    void *p = tsan_realloc(thr, pc, 0, 100);
     EXPECT_NE(p, (void*)0);
     memset(p, 0xde, 100);
-    user_free(thr, pc, p);
+    tsan_free(thr, pc, p);
   }
   {
-    void *p = user_alloc(thr, pc, 100);
+    void *p = tsan_malloc(thr, pc, 100);
     EXPECT_NE(p, (void*)0);
     memset(p, 0xde, 100);
-    void *p2 = user_realloc(thr, pc, p, 0);
-    EXPECT_NE(p2, (void*)0);
+    // Realloc(P, 0) is equivalent to free(N) and returns NULL.
+    void *p2 = tsan_realloc(thr, pc, p, 0);
+    EXPECT_EQ(p2, (void*)0);
   }
   {
-    void *p = user_realloc(thr, pc, 0, 100);
+    void *p = tsan_realloc(thr, pc, 0, 100);
     EXPECT_NE(p, (void*)0);
     memset(p, 0xde, 100);
-    void *p2 = user_realloc(thr, pc, p, 10000);
+    void *p2 = tsan_realloc(thr, pc, p, 10000);
     EXPECT_NE(p2, (void*)0);
     for (int i = 0; i < 100; i++)
       EXPECT_EQ(((char*)p2)[i], (char)0xde);
     memset(p2, 0xde, 10000);
-    user_free(thr, pc, p2);
+    tsan_free(thr, pc, p2);
   }
   {
-    void *p = user_realloc(thr, pc, 0, 10000);
+    void *p = tsan_realloc(thr, pc, 0, 10000);
     EXPECT_NE(p, (void*)0);
     memset(p, 0xde, 10000);
-    void *p2 = user_realloc(thr, pc, p, 10);
+    void *p2 = tsan_realloc(thr, pc, p, 10);
     EXPECT_NE(p2, (void*)0);
     for (int i = 0; i < 10; i++)
       EXPECT_EQ(((char*)p2)[i], (char)0xde);
-    user_free(thr, pc, p2);
+    tsan_free(thr, pc, p2);
   }
 }
 
 TEST(Mman, UsableSize) {
   ThreadState *thr = cur_thread();
   uptr pc = 0;
-  char *p = (char*)user_alloc(thr, pc, 10);
-  char *p2 = (char*)user_alloc(thr, pc, 20);
-  EXPECT_EQ(0U, user_alloc_usable_size(NULL));
-  EXPECT_EQ(10U, user_alloc_usable_size(p));
-  EXPECT_EQ(20U, user_alloc_usable_size(p2));
-  user_free(thr, pc, p);
-  user_free(thr, pc, p2);
-  EXPECT_EQ(0U, user_alloc_usable_size((void*)0x4123));
+  char *p = (char*)tsan_malloc(thr, pc, 10);
+  char *p2 = (char*)tsan_malloc(thr, pc, 20);
+  EXPECT_EQ(0U, tsan_alloc_usable_size(NULL));
+  EXPECT_EQ(10U, tsan_alloc_usable_size(p));
+  EXPECT_EQ(20U, tsan_alloc_usable_size(p2));
+  tsan_free(thr, pc, p);
+  tsan_free(thr, pc, p2);
+  EXPECT_EQ(0U, tsan_alloc_usable_size((void*)0x4123));
 }
 
 TEST(Mman, Stats) {
   ThreadState *thr = cur_thread();
 
   uptr alloc0 = __sanitizer_get_current_allocated_bytes();
   uptr heap0 = __sanitizer_get_heap_size();
   uptr free0 = __sanitizer_get_free_bytes();
   uptr unmapped0 = __sanitizer_get_unmapped_bytes();
 
   EXPECT_EQ(10U, __sanitizer_get_estimated_allocated_size(10));
   EXPECT_EQ(20U, __sanitizer_get_estimated_allocated_size(20));
   EXPECT_EQ(100U, __sanitizer_get_estimated_allocated_size(100));
 
-  char *p = (char*)user_alloc(thr, 0, 10);
+  char *p = (char*)tsan_malloc(thr, 0, 10);
   EXPECT_TRUE(__sanitizer_get_ownership(p));
   EXPECT_EQ(10U, __sanitizer_get_allocated_size(p));
 
   EXPECT_EQ(alloc0 + 16, __sanitizer_get_current_allocated_bytes());
   EXPECT_GE(__sanitizer_get_heap_size(), heap0);
   EXPECT_EQ(free0, __sanitizer_get_free_bytes());
   EXPECT_EQ(unmapped0, __sanitizer_get_unmapped_bytes());
 
-  user_free(thr, 0, p);
+  tsan_free(thr, 0, p);
 
   EXPECT_EQ(alloc0, __sanitizer_get_current_allocated_bytes());
   EXPECT_GE(__sanitizer_get_heap_size(), heap0);
   EXPECT_EQ(free0, __sanitizer_get_free_bytes());
   EXPECT_EQ(unmapped0, __sanitizer_get_unmapped_bytes());
 }
 
-TEST(Mman, CallocOverflow) {
-#if SANITIZER_DEBUG
-  // EXPECT_DEATH clones a thread with 4K stack,
-  // which is overflown by tsan memory accesses functions in debug mode.
-  return;
-#endif
+TEST(Mman, Valloc) {
+  ThreadState *thr = cur_thread();
+
+  void *p = tsan_valloc(thr, 0, 100);
+  EXPECT_NE(p, (void*)0);
+  tsan_free(thr, 0, p);
+
+  p = tsan_pvalloc(thr, 0, 100);
+  EXPECT_NE(p, (void*)0);
+  tsan_free(thr, 0, p);
+
+  p = tsan_pvalloc(thr, 0, 0);
+  EXPECT_NE(p, (void*)0);
+  EXPECT_EQ(GetPageSizeCached(), __sanitizer_get_allocated_size(p));
+  tsan_free(thr, 0, p);
+}
+
+#if !SANITIZER_DEBUG
+
+TEST(Mman, CallocOverflow) {
   ThreadState *thr = cur_thread();
   uptr pc = 0;
   size_t kArraySize = 4096;
   volatile size_t kMaxSizeT = std::numeric_limits<size_t>::max();
   volatile size_t kArraySize2 = kMaxSizeT / kArraySize + 10;
   volatile void *p = NULL;
-  EXPECT_DEATH(p = user_calloc(thr, pc, kArraySize, kArraySize2),
+  EXPECT_DEATH(p = tsan_calloc(thr, pc, kArraySize, kArraySize2),
+               "allocator is terminating the process instead of returning 0");
+  EXPECT_EQ(0L, p);
+}
+
+TEST(Mman, Memalign) {
+  ThreadState *thr = cur_thread();
+
+  void *p = tsan_memalign(thr, 0, 8, 100);
+  EXPECT_NE(p, (void*)0);
+  tsan_free(thr, 0, p);
+
+  p = NULL;
+  EXPECT_DEATH(p = tsan_memalign(thr, 0, 7, 100),
                "allocator is terminating the process instead of returning 0");
   EXPECT_EQ(0L, p);
 }
 
+TEST(Mman, PosixMemalign) {
+  ThreadState *thr = cur_thread();
+
+  void *p = NULL;
+  int res = tsan_posix_memalign(thr, 0, &p, 8, 100);
+  EXPECT_NE(p, (void*)0);
+  EXPECT_EQ(res, 0);
+  tsan_free(thr, 0, p);
+
+  p = NULL;
+  // Alignment is not a power of two, although is a multiple of sizeof(void*).
+  EXPECT_DEATH(res = tsan_posix_memalign(thr, 0, &p, 3 * sizeof(p), 100),
+               "allocator is terminating the process instead of returning 0");
+  EXPECT_EQ(0L, p);
+  // Alignment is not a multiple of sizeof(void*), although is a power of 2.
+  EXPECT_DEATH(res = tsan_posix_memalign(thr, 0, &p, 2, 100),
+               "allocator is terminating the process instead of returning 0");
+  EXPECT_EQ(0L, p);
+}
+
+TEST(Mman, AlignedAlloc) {
+  ThreadState *thr = cur_thread();
+
+  void *p = tsan_aligned_alloc(thr, 0, 8, 64);
+  EXPECT_NE(p, (void*)0);
+  tsan_free(thr, 0, p);
+
+  p = NULL;
+  // Alignement is not a power of 2.
+  EXPECT_DEATH(p = tsan_aligned_alloc(thr, 0, 7, 100),
+               "allocator is terminating the process instead of returning 0");
+  EXPECT_EQ(0L, p);
+  // Size is not a multiple of alignment.
+  EXPECT_DEATH(p = tsan_aligned_alloc(thr, 0, 8, 100),
+               "allocator is terminating the process instead of returning 0");
+  EXPECT_EQ(0L, p);
+}
+
+#endif
+
 }  // namespace __tsan

test/tsan/allocator_returns_null.cc

 // RUN:   | FileCheck %s --check-prefix=CHECK-nCRASH
 // RUN: %env_tsan_opts=allocator_may_return_null=1 not %run %t new 2>&1 \
 // RUN:   | FileCheck %s --check-prefix=CHECK-nCRASH
 // RUN: %env_tsan_opts=allocator_may_return_null=0 not %run %t new-nothrow 2>&1 \
 // RUN:   | FileCheck %s --check-prefix=CHECK-nnCRASH
 // RUN: %env_tsan_opts=allocator_may_return_null=1     %run %t new-nothrow 2>&1 \
 // RUN:   | FileCheck %s --check-prefix=CHECK-nnNULL
 
+// Until we figure out why errno is not properly set on windows
+// UNSUPPORTED: win32
+
 #include <assert.h>
-#include <string.h>
+#include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <string.h>
 #include <limits>
 #include <new>
 
 int main(int argc, char **argv) {
   // Disable stderr buffering. Needed on Windows.
   setvbuf(stderr, NULL, _IONBF, 0);
 
   assert(argc == 2);
   const char *action = argv[1];
   fprintf(stderr, "%s:\n", action);
 
+  // The limit enforced in tsan_mman.cc, user_alloc function.
   static const size_t kMaxAllowedMallocSizePlusOne = (1ULL << 40) + 1;
 
   void *x = 0;
   if (!strcmp(action, "malloc")) {
     x = malloc(kMaxAllowedMallocSizePlusOne);
   } else if (!strcmp(action, "calloc")) {
     x = calloc((kMaxAllowedMallocSizePlusOne / 4) + 1, 4);
   } else if (!strcmp(action, "calloc-overflow")) {
int main(int argc, char **argv) {
   } else if (!strcmp(action, "new")) {
     x = operator new(kMaxAllowedMallocSizePlusOne);
   } else if (!strcmp(action, "new-nothrow")) {
     x = operator new(kMaxAllowedMallocSizePlusOne, std::nothrow);
   } else {
     assert(0);
   }
 
+  fprintf(stderr, "errno: %d\n", errno);
+
   // The NULL pointer is printed differently on different systems, while (long)0
   // is always the same.
   fprintf(stderr, "x: %lx\n", (long)x);
   free(x);
+
   return x != 0;
 }
 
 // CHECK-mCRASH: malloc:
 // CHECK-mCRASH: ThreadSanitizer's allocator is terminating the process
 // CHECK-cCRASH: calloc:
 // CHECK-cCRASH: ThreadSanitizer's allocator is terminating the process
 // CHECK-coCRASH: calloc-overflow:
 // CHECK-coCRASH: ThreadSanitizer's allocator is terminating the process
 // CHECK-rCRASH: realloc:
 // CHECK-rCRASH: ThreadSanitizer's allocator is terminating the process
 // CHECK-mrCRASH: realloc-after-malloc:
 // CHECK-mrCRASH: ThreadSanitizer's allocator is terminating the process
 // CHECK-nCRASH: new:
 // CHECK-nCRASH: ThreadSanitizer's allocator is terminating the process
 // CHECK-nnCRASH: new-nothrow:
 // CHECK-nnCRASH: ThreadSanitizer's allocator is terminating the process
 
 // CHECK-mNULL: malloc:
+// CHECK-mNULL: errno: 12
 // CHECK-mNULL: x: 0
 // CHECK-cNULL: calloc:
+// CHECK-cNULL: errno: 12
 // CHECK-cNULL: x: 0
 // CHECK-coNULL: calloc-overflow:
+// CHECK-coNULL: errno: 12
 // CHECK-coNULL: x: 0
 // CHECK-rNULL: realloc:
+// CHECK-rNULL: errno: 12
 // CHECK-rNULL: x: 0
 // CHECK-mrNULL: realloc-after-malloc:
+// CHECK-mrNULL: errno: 12
 // CHECK-mrNULL: x: 0
 // CHECK-nnNULL: new-nothrow:
 // CHECK-nnNULL: x: 0