This is an archive of the discontinued LLVM Phabricator instance.

[esan] Intercept calloc to avoid deadlocks with tcmalloc
ClosedPublic

Authored by bruening on Jun 7 2016, 10:39 AM.

Details

Summary

When tcmalloc initializes before esan, esan's initialization ends up
calling back into tcmalloc due to the calloc done by dlsym. This results
in a deadlock. We avoid this by special-casing this single allocation.

Intercepting calloc also gives us the opportunity to act on its zeroing as
stores by the application.

Diff Detail

Event Timeline

bruening updated this revision to Diff 59911.Jun 7 2016, 10:39 AM
bruening retitled this revision from to [esan] Intercept calloc to avoid deadlocks with tcmalloc.
bruening updated this object.
bruening added a reviewer: aizatsky.
bruening added subscribers: llvm-commits, eugenis, kcc and 2 others.
aizatsky requested changes to this revision.Jun 7 2016, 1:46 PM
aizatsky edited edge metadata.
aizatsky added inline comments.
lib/esan/esan_interceptors.cpp
431

Is there a multi-threading concern here? I assume tcmalloc initializes in one thread only?

450

Do you want to clear used_early_alloc_buf here?

This revision now requires changes to proceed.Jun 7 2016, 1:46 PM
bruening added inline comments.Jun 7 2016, 3:22 PM
lib/esan/esan_interceptors.cpp
431

This will only happen at process init prior to __esan_init, but I suppose it's possible for some shared library to create early threads that all race here. Given the fragility of sanitizer libc interceptors in general I'd have to say that's not high on my list of corner cases to worry about. Are you ok with a comment saying we simply don't handle it?

450

Sure, for cleanliness, though we do not expect it to matter (we expect a single call to calloc during dlsym prior to REAL(calloc) being set and after that to never need the buffer again).

bruening updated this revision to Diff 59963.Jun 7 2016, 3:23 PM
bruening edited edge metadata.

Clear the used flag, and comment on races.

aizatsky accepted this revision.Jun 7 2016, 3:30 PM
aizatsky edited edge metadata.
This revision is now accepted and ready to land.Jun 7 2016, 3:30 PM
This revision was automatically updated to reflect the committed changes.