This is an archive of the discontinued LLVM Phabricator instance.

[scan-build] Fix deadlock at failures in libears/ear.c
ClosedPublic

Authored by steakhal on Jan 28 2022, 12:42 AM.

Details

Summary

We experienced some deadlocks when we used multiple threads for logging
using scan-builds intercept-build tool when we used multiple threads by
e.g. logging make -j16

(gdb) bt
#0  0x00007f2bb3aff110 in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007f2bb3af70a3 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007f2bb3d152e4 in ?? ()
#3  0x00007ffcc5f0cc80 in ?? ()
#4  0x00007f2bb3d2bf5b in ?? () from /lib64/ld-linux-x86-64.so.2
#5  0x00007f2bb3b5da27 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#6  0x00007f2bb3b5dbe0 in exit () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x00007f2bb3d144ee in ?? ()
#8  0x746e692f706d742f in ?? ()
#9  0x692d747065637265 in ?? ()
#10 0x2f653631326b3034 in ?? ()
#11 0x646d632e35353532 in ?? ()
#12 0x0000000000000000 in ?? ()

I think the gcc's exit call caused the injected libear.so to be unloaded
by the ld, which in turn called the void on_unload() __attribute__((destructor)).
That tried to acquire an already locked mutex which was left locked in the
bear_report_call() call, that probably encountered some error and
returned early when it forgot to unlock the mutex.

All of these are speculation since from the backtrace I could not verify
if frames 2 and 3 are in fact corresponding to the libear.so module.
But I think it's a fairly safe bet.

So, hereby I'm releasing the held mutex on *all paths*, even if some failure
happens.

PS: I would use lock_guards, but it's C.

Diff Detail

Event Timeline

steakhal created this revision.Jan 28 2022, 12:42 AM
steakhal requested review of this revision.Jan 28 2022, 12:42 AM

It would be quite difficult to write a test for this issue, so I decided not to.

NoQ accepted this revision.Jan 28 2022, 11:30 AM

Fair enough!

This revision is now accepted and ready to land.Jan 28 2022, 11:30 AM

@tstellar, do you think it worth backporting to 13.0.1?

This revision was landed with ongoing or failed builds.Feb 2 2022, 3:56 AM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptFeb 2 2022, 3:56 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript

We missed the release, let's backport it to 14.0.0.