More optimizations:

BM_youtube/min_time:15.000/repeats:4/manual_time_mean       410274 us       615999 us            4
BM_youtube/min_time:15.000/repeats:4/manual_time_median     410246 us       616125 us            4
BM_youtube/min_time:15.000/repeats:4/manual_time_stddev        302 us          587 us            4

It's OK with me. Part of the reason for splitting out the symbolizer was some memory overhead as well, which we probably want to look into.
Is COMPILER_RT_HAS_GWP_ASAN going to be true by default for Android? If so we probably should loop in the toolchain team so that there is no surprise on their next toolchain update.

Addressing review comments:

• commenting the use of the #pragma nounroll
• using the platform specific types in mutex.h

Move the mutex state enum into linux.cc since it's only used there.

• Adding constexpr for the number of yields & trys in the mutex class;
• Adding a test to ensure that scudo::getPageSizeCached is consitent with getpagesize;

Regarding Teresa's issue, it seems to boil down to the version of clang, and not inlining the __atomic_load_8 & co.
Looking at godbolt, it doesn't seem to happen until clang 6, while gcc did it since a while.
After attempting to conditionally add -latomic, and failing, I am resorting to adding -latomic unconditionally to the tests link flags, unless someone has a better idea.

Tentative fix for this is in https://reviews.llvm.org/D64086, if you could please let me know if that works for you.

s/kostya/kostyak/

clang-format'd.

Thank you Douglas!

Remove unchanged files from CL.

Adding a testcase for a Class to the New tests.

Changing some errno behaviors to be more consistent.
Renaming wrappers.inc to wrappers_c.inc and updated associated strings.

Fixing a couple of errors that came up in testing.

Removing a reinterpret_cast<void *>.

Addressing review comments:

• renaming allocator.h to allocator_config.h
• adding FuchsiaConfig to the battery of tests (64-bit only)

Tune some tests & Quarantine parameters.

Enabled the Quarantine for some tests, which wasn't used before.

Update the deleters to actually delete the pointer itself as well.

Always call initOnce, and rename it initOnceMaybe to reflect that
initialization might not necessarily occur if it already happened.

clang-format'ing the code.

Use an atomic_u8 for Initialized instead of a bool.

As discussed through comments, re-introduce a StaticSpinMutex to the
"once" initialization of the TSD registry, and remove its no-op
initLinkerInitialized in the various places it was used.
Also adds a test to exercise the path of "direct" initialization
via calling initLinkerInitialized directly on the registry.

Ping pretty please! I think all the comments were addressed one way or another. The new structure doesn't "cycle" anymore.

I think it's good from the integration perspective.
The only potential concern I have is the specific use of the TLS variables in GWP-ASan which might not work everywhere here (eg: if a platform is using emutls as TLS as it uses malloc() internally).
Have you tested that locally with an Android emulator?

Disable a test on 32-bit for now: we are running out of address space.
This will be fixed in a subsequent CL, but affects the tests of this one.

Change the Primary test again to be more forgiving to OOM.

Add a comment for MinimalInit.

This, hopefully, detangles a bit the initialization process for
the registry. In order to do that, we get rid of the mutex in favor
of our own call_once type construct (losely inspired by
llvm::call_once). We move the initialization code into
initLinkerInitialized.

Switching from a spin mutex to a blocking mutex for initOnce.

Regarding the Scudo side of the patch: the code has to be able to compile with gcc as well, and not necessarily the latest version.
This won't compile on systems without a signature_HYGON_*.

Addressing several of Matt's review points:

• adding comments to obscure code snippets
• calling initLinkerInitialized when needed
• changing unique_ptr's get()-> to ->
• simplifying some code constructs

Matt, thank you for all the reviews you are doing. Very insightful points.

Correct a test and some formatting.

Correcting a typo (meaningul -> meaningful)

Adding a forgotten CHECK as well.

Correcting some code constructs as suggested by Matt:

• using unique_ptr in tests;
• accounting for a potential null TransferBatch in refill;
• other corrections.