Use pthreads for thread-local lsan allocator cache on darwin

[fjricci]

Summary

This patch allows us to move away from using __thread on darwin,
which is requiring for building lsan for darwin on ios version 7
and on iossim i386.

[kubamracek]

lib/lsan/lsan_allocator.cc

29 ↗

(On Diff #88566)

Again, why do we need this change? We need to make sure not to break these platforms (mips, aarch64).

111 ↗

(On Diff #88566)

Looks too weird to me. sizeof(AllocatorCache) should do.

lib/lsan/lsan_allocator.h

58 ↗

(On Diff #88566)

Do we need to change this code here? It's not obvious to me that this doesn't change the parameters for some platforms.

lib/sanitizer_common/sanitizer_allocator_primary64.h

49–50 ↗

(On Diff #88566)

Why is this necessary? I mean, the change looks reasonable, but is it related to this change? If not, please submit it as a separate patch.

[fjricci]

lib/lsan/lsan_allocator.cc

29 ↗

(On Diff #88566)

This is exactly equivalent to the existing #if condition (#if (mips || aarch || i386) && !i386), but simplified because the data has been split out.

111 ↗

(On Diff #88566)

sizeof(TYPENAME) fails check-sanitizer style checks.

lib/lsan/lsan_allocator.h

58 ↗

(On Diff #88566)

It seemed cleaner to do this way, now that kMaxAllowedMallocSize has its own if block. However, I think you're right, this could potentially miss some edge cases for architecture.

lib/sanitizer_common/sanitizer_allocator_primary64.h

49–50 ↗

(On Diff #88566)

It's necessary, at least on Darwin, because you'll overflow the type if you use a uptr to store a ULL. I'll move it to a separate patch.

[kubamracek]

lib/lsan/lsan_allocator.cc

29 ↗

(On Diff #88566)

I'd still prefer not to make refactoring together with functional changes. (This particular change is okay, but it's hard to immediately see that the change is NFC.)

111 ↗

(On Diff #88566)

Does it really? Is the linter smart enough to figure out that AllocatorCache is a type and not a variable? Any other suggestions? I really don't like to have a function call inside a sizeof() (I know it's not really calling the function, but still.)

lib/sanitizer_common/sanitizer_allocator_primary64.h

49–50 ↗

(On Diff #88566)

This whole class should be ifdef'd to only compile on 64-bit platforms. Isn't that so?

[fjricci]

lib/lsan/lsan_allocator.cc

111 ↗

(On Diff #88566)

Ahh, yeah, the linter isn't smart enough.

lib/sanitizer_common/sanitizer_allocator_primary64.h

49–50 ↗

(On Diff #88566)

On non-windows platforms, uptr is defined as an unsigned long, not an unsigned long long (sanitizer_internal_defs.h:103)

[kubamracek]

lib/sanitizer_common/sanitizer_allocator_primary64.h

49–50 ↗

(On Diff #88566)

Right, and unsigned long is 64-bit on (supported) 64-bit platforms, isn't it?

[fjricci]

lib/sanitizer_common/sanitizer_allocator_primary64.h

49–50 ↗

(On Diff #88566)

Turns out that __arm__ was creeping in due to a gap in the if condition. I'll fix that.

[fjricci]

Fix architecture issues

[fjricci]

Rebase

[fjricci]

Rebase and initialize AllocatorCache

[kubamracek]

lib/lsan/lsan_common.h

18 ↗

(On Diff #88841)

Hm... Could we have it the other way around? lsan_allocator.h would include lsan_common.h and GetAllocatorCache would be declared in lsan_allocator.h only. lsan_allocator.cc would include lsan_allocator.h. The _linux.cc and _mac.cc files would also include lsan_allocator.h. Is there a dependency problem with that?

[fjricci]

lib/lsan/lsan_common.h

18 ↗

(On Diff #88841)

Ahh, that's a very good idea, didn't think to move just the declaration.

[fjricci]

Move declaration of GetAllocatorCache

[kubamracek]

lib/lsan/lsan_allocator.h

55 ↗

(On Diff #88846)

Does the issue with __arm__ still exist now?

[fjricci]

Remove arm workaround

[kubamracek]

LGTM. Thanks for working on this and for being patient with the review!

[fjricci]

Thanks for taking the time to review! This should fix the last of the build failures, currently working on a patch to enable builds by default but disable actually running it.

[kubamracek]

In D29994#679582, @fjricci wrote:

Thanks for taking the time to review! This should fix the last of the build failures, currently working on a patch to enable builds by default but disable actually running it.

Okay. It should be enough to exclude "check-lsan" from "check-all", but still keep "check-lsan" so that we can run the tests manually. Look for add_lit_testsuite and EXCLUDE_FROM_ALL.

[fjricci]

In D29994#679588, @kubamracek wrote:

In D29994#679582, @fjricci wrote:

Thanks for taking the time to review! This should fix the last of the build failures, currently working on a patch to enable builds by default but disable actually running it.

Okay. It should be enough to exclude "check-lsan" from "check-all", but still keep "check-lsan" so that we can run the tests manually. Look for add_lit_testsuite and EXCLUDE_FROM_ALL.

We'll also need to change the default in asan, since asan enables lsan by default currently.

[fjricci]

This patch causes a build failure on one test on linux, and I'm having trouble figuring out why. It appears that the test error is fixed by simply moving the definition of GetAllocatorCache() from lsan_common_linux.cc to lsan_allocator.cc. Not sure why this would change the behavior of the test, but investigating now. In the meantime, I reverted the patch.

[fjricci]

The failing test is Linux-x86_64 stack-use-after-return.cc, failing with: AddressSanitizer: nested bug in the same thread, aborting.

[kubamracek]

Can you post the backtrace of the crash/failure?

[fjricci]

FAIL: AddressSanitizer-x86_64-linux :: TestCases/Posix/stack-use-after-return.cc (1102 of 1258)

• TEST 'AddressSanitizer-x86_64-linux :: TestCases/Posix/stack-use-after-return.cc' FAILED ****

Script:

/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -O0 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc -pthread -o /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp && env ASAN_OPTIONS=detect_stack_use_after_return=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -O1 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc -pthread -o /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp && env ASAN_OPTIONS=detect_stack_use_after_return=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -O2 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc -pthread -o /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp && env ASAN_OPTIONS=detect_stack_use_after_return=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -O3 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc -pthread -o /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp && env ASAN_OPTIONS=detect_stack_use_after_return=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc
env ASAN_OPTIONS=detect_stack_use_after_return=0 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -O3 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc -pthread -o /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp -DkSize=10000 -DUseThread -DkStackSize=65536 && env ASAN_OPTIONS=detect_stack_use_after_return=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck --check-prefix=THREAD /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -DUseThread -O2 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc -pthread -o /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp && env ASAN_OPTIONS=detect_stack_use_after_return=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck --check-prefix=THREAD /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc
env ASAN_OPTIONS=detect_stack_use_after_return=1:max_uar_stack_size_log=20:verbosity=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck --check-prefix=CHECK-20 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc

env ASAN_OPTIONS=detect_stack_use_after_return=1:min_uar_stack_size_log=24:max_uar_stack_size_log=24:verbosity=1 not /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm_build_ninja/projects/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Posix/Output/stack-use-after-return.cc.tmp 2>&1 | FileCheck --check-prefix=CHECK-24 /mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc

Exit Code: 1

Command Output (stderr):

1: 0x7fff258045e0
2: 0x7fff258045e0
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/asan/TestCases/Posix/stack-use-after-return.cc:52:13: error: expected string not found in input
// THREAD: WRITE of size 1 {{.*}} thread T{{[1-9]}}

^

<stdin>:1:1: note: scanning from here
1: 0x7f922b0cf820
^
<stdin>:5:24: note: possible intended match here
AddressSanitizer: nested bug in the same thread, aborting.

^

--

[kubamracek]

Right, but can you run this in a debugger and see what the actual backtrace is?

[fjricci]

I believe the issue is that we're using too small of a stack in that test, and perhaps we're just pushed over the limit by this new AllocatorCache. If I change -DkStackSize=65536 to be a number >=75392, the test passes. Is this an acceptable solution? If so, what would be a good number to use?

[kubamracek]

In D29994#683576, @fjricci wrote:

I believe the issue is that we're using too small of a stack in that test, and perhaps we're just pushed over the limit by this new AllocatorCache. If I change -DkStackSize=65536 to be a number >=75392, the test passes. Is this an acceptable solution? If so, what would be a good number to use?

Ok, that's even weirder. Why should this change the local stack overhead? Do we overflow the stack? Can you actually post the backtrace from the debugger? Why is there a nested ASan failure?

[fjricci]

The stack trace from the debugger isn't particularly helpful:

 thread #2, name = 'stack-use-after', stop reason = signal SIGSEGV: address access protected (fault address: 0x7ffff7e72020)
  frame #0: stack-use-after-return.cc.tmp`Func1() at stack-use-after-return.cc:39
   36  	}
   37  	
   38  	__attribute__((noinline))
-> 39  	char *Func1() {
   40  	  char local[kSize];
   41  	  return Ident(local);
   42  	}
(lldb) bt all
  thread #1, name = 'stack-use-after'
    frame #0: libpthread.so.0`pthread_join(threadid=140737352574720, thread_return=0x0000000000000000) at pthread_join.c:92
    frame #1: stack-use-after-return.cc.tmp`main at stack-use-after-return.cc:74
    frame #2: libc.so.6`__libc_start_main(main=(stack-use-after-return.cc.tmp`main), argc=1, argv=0x00007fffffffdb78, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffdb68) at libc-start.c:287
    frame #3: 0x000000000041ba7c stack-use-after-return.cc.tmp`_start + 41
* thread #2, name = 'stack-use-after', stop reason = signal SIGSEGV: address access protected (fault address: 0x7ffff7e72020)
  * frame #0: stack-use-after-return.cc.tmp`Func1() at stack-use-after-return.cc:39
    frame #1: stack-use-after-return.cc.tmp`Thread(void*) at stack-use-after-return.cc:61
    frame #2: libpthread.so.0`start_thread(arg=0x00007ffff7e82700) at pthread_create.c:312
    frame #3: libc.so.6`__clone at clone.S:111

The crash error also goes away if I add a CHECK(0); to GetAllocatorCache(), which is similarly strange.

[fjricci]

That does look like it could be a stack-overflow seg fault though.

[fjricci]

I found something very odd and very relevant. I added a line in test-use-after-return.cc to call pthread_attr_getstacksize immediately after pthread_attr_setstacksize. On master, the returned stack size is 8388608. With my change, it's 65536. I assume this means that we're intercepting something inside call to pthread_attr_setstacksize. But regardless, this means that 65536 was not large enough to begin with, unless we expect the sanitizers to intercept that call and increase the stack size. I assume we don't, since there's no direct interceptor for pthread_attr_setstacksize.

My diff is:

int main(int argc, char **argv) {
   pthread_attr_init(&attr);
   if (kStackSize > 0)
     pthread_attr_setstacksize(&attr, kStackSize);
+  size_t stacksize = 0;
+  pthread_attr_getstacksize(&attr, &stacksize);
+  printf("stacksize: %lu\n", stacksize);
   pthread_t t;
   pthread_create(&t, &attr, Thread, 0);
   pthread_attr_destroy(&attr);

[kubamracek]

Ok, then the test is just flaky. Let's simply double the stack size value in the test (as a separate commit).

[fjricci]

Reverted due to failure with D30267

[fjricci]

This still fails on aarch64, even with the recent work by @kubamracek:

http://lab.llvm.org:8011/builders/clang-cmake-aarch64-42vma/builds/5723/steps/ninja%20check%201/logs/FAIL%3A%20AddressSanitizer-aarch64-linux%3A%3Astack-use-after-return.cc

[kubamracek]

Okay, this is getting ridiculous. I've disabled the test on AArch64 in r298267. The test was not testing the right thing on AArch64 anyway (it wasn't setting the stack size).

[fjricci]

This patch is having a real rough time. @vitalybuka thinks it probably caused this failure, and I wouldn't be too surprised: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/1117/steps/test%20standalone%20compiler-rt/logs/stdio

It doesn't repro on my CentOS or my Ubuntu machine though...

[fjricci]

Actually, looks like the buildbot isn't failing currently. I won't revert the patch for now.

[fjricci]

Going to try to get the failure to repro locally.

[fjricci]

I've tried everything I can think of in terms of a repro here - different gcc/clang versions, etc. I also looked up the zorg config and ran exactly the way it was run on the buildbot (standalone build + i386 test), and still no failure. Perhaps I can split this into two changes, to try to see where the failure is springing up.

My guess is that this is still caused simply by moving the GetAllocatorCache() to a different file, so I can try to commit the whole change minus that bit first, and make sure tests pass. If just moving the function definition to a different file causes a failure, it's very unlikely to me that the failure is legitimate and not a flake.

Given that the code review on this is fine, I'll go ahead with merging the partial commit (the bit that moves the definitions to the header and adds a function for GetAllocatorCache() in the same file), provided I don't hear any objection. I'll then upload another review for the second half.

[fjricci]

Split revision into smaller pieces

[fjricci]

Actually, let's make it even smaller.

compiler-rt/trunk/lib/lsan/lsan_allocator.h

 // This file is a part of LeakSanitizer.
 // Allocator for standalone LSan.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LSAN_ALLOCATOR_H
 #define LSAN_ALLOCATOR_H
 
+#include "sanitizer_common/sanitizer_allocator.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_internal_defs.h"
+#include "lsan_common.h"
 
 namespace __lsan {
 
 void *Allocate(const StackTrace &stack, uptr size, uptr alignment,
                bool cleared);
 void Deallocate(void *p);
 void *Reallocate(const StackTrace &stack, void *p, uptr new_size,
                  uptr alignment);
 uptr GetMallocUsableSize(const void *p);
 
 template<typename Callable>
 void ForEachChunk(const Callable &callback);
 
 void GetAllocatorCacheRange(uptr *begin, uptr *end);
 void AllocatorThreadFinish();
 void InitializeAllocator();
 
+struct ChunkMetadata {
+  u8 allocated : 8;  // Must be first.
+  ChunkTag tag : 2;
+#if SANITIZER_WORDSIZE == 64
+  uptr requested_size : 54;
+#else
+  uptr requested_size : 32;
+  uptr padding : 22;
+#endif
+  u32 stack_trace_id;
+};
+
+#if defined(__mips64) || defined(__aarch64__) || defined(__i386__)
+static const uptr kRegionSizeLog = 20;
+static const uptr kNumRegions = SANITIZER_MMAP_RANGE_SIZE >> kRegionSizeLog;
+typedef TwoLevelByteMap<(kNumRegions >> 12), 1 << 12> ByteMap;
+typedef CompactSizeClassMap SizeClassMap;
+typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE,
+    sizeof(ChunkMetadata), SizeClassMap, kRegionSizeLog, ByteMap>
+    PrimaryAllocator;
+#elif defined(__x86_64__)
+struct AP64 {  // Allocator64 parameters. Deliberately using a short name.
+  static const uptr kSpaceBeg = 0x600000000000ULL;
+  static const uptr kSpaceSize =  0x40000000000ULL; // 4T.
+  static const uptr kMetadataSize = sizeof(ChunkMetadata);
+  typedef DefaultSizeClassMap SizeClassMap;
+  typedef NoOpMapUnmapCallback MapUnmapCallback;
+  static const uptr kFlags = 0;
+};
+
+typedef SizeClassAllocator64<AP64> PrimaryAllocator;
+#endif
+typedef SizeClassAllocatorLocalCache<PrimaryAllocator> AllocatorCache;
+
+AllocatorCache *GetAllocatorCache();
 }  // namespace __lsan
 
 #endif  // LSAN_ALLOCATOR_H

compiler-rt/trunk/lib/lsan/lsan_allocator.cc

 #include "sanitizer_common/sanitizer_internal_defs.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
 #include "sanitizer_common/sanitizer_stacktrace.h"
 #include "lsan_common.h"
 
 extern "C" void *memset(void *ptr, int value, uptr num);
 
 namespace __lsan {
-
-struct ChunkMetadata {
-  u8 allocated : 8;  // Must be first.
-  ChunkTag tag : 2;
-#if SANITIZER_WORDSIZE == 64
-  uptr requested_size : 54;
-#else
-  uptr requested_size : 32;
-  uptr padding : 22;
-#endif
-  u32 stack_trace_id;
-};
-
-#if defined(__mips64) || defined(__aarch64__) || defined(__i386__)
 #if defined(__i386__)
 static const uptr kMaxAllowedMallocSize = 1UL << 30;
-#else
+#elif defined(__mips64) || defined(__aarch64__)
 static const uptr kMaxAllowedMallocSize = 4UL << 30;
-#endif
-static const uptr kRegionSizeLog = 20;
-static const uptr kNumRegions = SANITIZER_MMAP_RANGE_SIZE >> kRegionSizeLog;
-typedef TwoLevelByteMap<(kNumRegions >> 12), 1 << 12> ByteMap;
-typedef CompactSizeClassMap SizeClassMap;
-typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE,
-    sizeof(ChunkMetadata), SizeClassMap, kRegionSizeLog, ByteMap>
-    PrimaryAllocator;
 #else
 static const uptr kMaxAllowedMallocSize = 8UL << 30;
-
-struct AP64 {  // Allocator64 parameters. Deliberately using a short name.
-  static const uptr kSpaceBeg = 0x600000000000ULL;
-  static const uptr kSpaceSize =  0x40000000000ULL; // 4T.
-  static const uptr kMetadataSize = sizeof(ChunkMetadata);
-  typedef DefaultSizeClassMap SizeClassMap;
-  typedef NoOpMapUnmapCallback MapUnmapCallback;
-  static const uptr kFlags = 0;
-};
-
-typedef SizeClassAllocator64<AP64> PrimaryAllocator;
 #endif
-typedef SizeClassAllocatorLocalCache<PrimaryAllocator> AllocatorCache;
 typedef LargeMmapAllocator<> SecondaryAllocator;
 typedef CombinedAllocator<PrimaryAllocator, AllocatorCache,
           SecondaryAllocator> Allocator;
 
 static Allocator allocator;
-static THREADLOCAL AllocatorCache cache;
 
 void InitializeAllocator() {
   allocator.InitLinkerInitialized(
       common_flags()->allocator_may_return_null,
       common_flags()->allocator_release_to_os_interval_ms);
 }
 
 void AllocatorThreadFinish() {
-  allocator.SwallowCache(&cache);
+  allocator.SwallowCache(GetAllocatorCache());
 }
 
 static ChunkMetadata *Metadata(const void *p) {
   return reinterpret_cast<ChunkMetadata *>(allocator.GetMetaData(p));
 }
 
 static void RegisterAllocation(const StackTrace &stack, void *p, uptr size) {
   if (!p) return;

 void *Allocate(const StackTrace &stack, uptr size, uptr alignment,
                bool cleared) {
   if (size == 0)
     size = 1;
   if (size > kMaxAllowedMallocSize) {
     Report("WARNING: LeakSanitizer failed to allocate %zu bytes\n", size);
     return nullptr;
   }
-  void *p = allocator.Allocate(&cache, size, alignment, false);
+  void *p = allocator.Allocate(GetAllocatorCache(), size, alignment, false);
   // Do not rely on the allocator to clear the memory (it's slow).
   if (cleared && allocator.FromPrimary(p))
     memset(p, 0, size);
   RegisterAllocation(stack, p, size);
   if (&__sanitizer_malloc_hook) __sanitizer_malloc_hook(p, size);
   RunMallocHooks(p, size);
   return p;
 }
 
 void Deallocate(void *p) {
   if (&__sanitizer_free_hook) __sanitizer_free_hook(p);
   RunFreeHooks(p);
   RegisterDeallocation(p);
-  allocator.Deallocate(&cache, p);
+  allocator.Deallocate(GetAllocatorCache(), p);
 }
 
 void *Reallocate(const StackTrace &stack, void *p, uptr new_size,
                  uptr alignment) {
   RegisterDeallocation(p);
   if (new_size > kMaxAllowedMallocSize) {
     Report("WARNING: LeakSanitizer failed to allocate %zu bytes\n", new_size);
-    allocator.Deallocate(&cache, p);
+    allocator.Deallocate(GetAllocatorCache(), p);
     return nullptr;
   }
-  p = allocator.Reallocate(&cache, p, new_size, alignment);
+  p = allocator.Reallocate(GetAllocatorCache(), p, new_size, alignment);
   RegisterAllocation(stack, p, new_size);
   return p;
 }
 
 void GetAllocatorCacheRange(uptr *begin, uptr *end) {
-  *begin = (uptr)&cache;
-  *end = *begin + sizeof(cache);
+  *begin = (uptr)GetAllocatorCache();
+  *end = *begin + sizeof(AllocatorCache);
 }
 
 uptr GetMallocUsableSize(const void *p) {
   ChunkMetadata *m = Metadata(p);
   if (!m) return 0;
   return m->requested_size;
 }
 

compiler-rt/trunk/lib/lsan/lsan_common_linux.cc

 
 #if CAN_SANITIZE_LEAKS && SANITIZER_LINUX
 #include <link.h>
 
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_flags.h"
 #include "sanitizer_common/sanitizer_linux.h"
 #include "sanitizer_common/sanitizer_stackdepot.h"
+#include "lsan_allocator.h"
 
 namespace __lsan {
 
 static const char kLinkerName[] = "ld";
 
 static char linker_placeholder[sizeof(LoadedModule)] ALIGNED(64);
 static LoadedModule *linker = nullptr;
 
 static bool IsLinker(const char* full_name) {
   return LibraryNameIs(full_name, kLinkerName);
 }
 
 static THREADLOCAL u32 current_thread_tid = kInvalidTid;
 u32 GetCurrentThread() { return current_thread_tid; }
 void SetCurrentThread(u32 tid) { current_thread_tid = tid; }
 
+static THREADLOCAL AllocatorCache allocator_cache;
+AllocatorCache *GetAllocatorCache() { return &allocator_cache; }
+
 __attribute__((tls_model("initial-exec")))
 THREADLOCAL int disable_counter;
 bool DisabledInThisThread() { return disable_counter > 0; }
 void DisableInThisThread() { disable_counter++; }
 void EnableInThisThread() {
   if (disable_counter == 0) {
     DisableCounterUnderflow();
   }

compiler-rt/trunk/lib/lsan/lsan_common_mac.cc

 //=-- lsan_common_mac.cc --------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of LeakSanitizer.
 // Implementation of common leak checking functionality. Darwin-specific code.
 //
 //===----------------------------------------------------------------------===//
 
-#include "sanitizer_common/sanitizer_allocator_internal.h"
 #include "sanitizer_common/sanitizer_platform.h"
 #include "lsan_common.h"
 
 #if CAN_SANITIZE_LEAKS && SANITIZER_MAC
 
+#include "sanitizer_common/sanitizer_allocator_internal.h"
+#include "lsan_allocator.h"
+
 #include <pthread.h>
 
 namespace __lsan {
 
 typedef struct {
   int disable_counter;
   u32 current_thread_id;
+  AllocatorCache cache;
 } thread_local_data_t;
 
 static pthread_key_t key;
 static pthread_once_t key_once = PTHREAD_ONCE_INIT;
 
 static void make_tls_key() { CHECK_EQ(pthread_key_create(&key, NULL), 0); }
 
 static thread_local_data_t *get_tls_val() {
   pthread_once(&key_once, make_tls_key);
 
   thread_local_data_t *ptr = (thread_local_data_t *)pthread_getspecific(key);
   if (ptr == NULL) {
     ptr = (thread_local_data_t *)InternalAlloc(sizeof(*ptr));
     ptr->disable_counter = 0;
     ptr->current_thread_id = kInvalidTid;
+    ptr->cache = AllocatorCache();
     pthread_setspecific(key, ptr);
   }
 
   return ptr;
 }
 
 bool DisabledInThisThread() { return get_tls_val()->disable_counter > 0; }
 
 void DisableInThisThread() { ++get_tls_val()->disable_counter; }
 
 void EnableInThisThread() {
   int *disable_counter = &get_tls_val()->disable_counter;
   if (*disable_counter == 0) {
     DisableCounterUnderflow();
   }
   --*disable_counter;
 }
 
 u32 GetCurrentThread() { return get_tls_val()->current_thread_id; }
 
 void SetCurrentThread(u32 tid) { get_tls_val()->current_thread_id = tid; }
 
+AllocatorCache *GetAllocatorCache() { return &get_tls_val()->cache; }
+
 void InitializePlatformSpecificModules() {
   CHECK(0 && "unimplemented");
 }
 
 // Scans global variables for heap pointers.
 void ProcessGlobalRegions(Frontier *frontier) {
   CHECK(0 && "unimplemented");
 }