tsan: switch to the new sanitizer_common mutex

[dvyukov]

Summary

Now that sanitizer_common mutex has feature-parity with tsan mutex,
switch tsan to the sanitizer_common mutex and remove tsan's custom mutex.

[vitalybuka]

"arc patch D106379" conflicts with origin/main

[dvyukov]

upload all 3 commits

[dvyukov]

This review contains 3 commits. I figured out that just the sanitizer_common part w/o a use example is not very useful for review, so added how tsan uses it as well.
I think Phabricator does not allow to review commit-per-commit, at least I did not find how to do it. So I uploaded this to github as well:
https://github.com/dvyukov/llvm-project/pull/5/commits

[vitalybuka]

In D106379#2891062, @dvyukov wrote:

This review contains 3 commits. I figured out that just the sanitizer_common part w/o a use example is not very useful for review, so added how tsan uses it as well.
I think Phabricator does not allow to review commit-per-commit, at least I did not find how to do it. So I uploaded this to github as well:
https://github.com/dvyukov/llvm-project/pull/5/commits

Can you just set entire "parent/child" chain in Phabricator?

[dvyukov]

I have more and more doubts re the thread safety annotations over time. They were kinda useful in a single particular case in new tsan runtime, where static analysis pointed precisely at unprotected accesses. But now we need more and more of NO_THREAD_SAFETY_ANALYSIS... maybe we should drop it?

[dvyukov]

"arc patch D106379" conflicts with origin/main

I hope this is fixed after I pushed all 3 commits.

Can you just set entire "parent/child" chain in Phabricator?

Where/how do I do this?

[vitalybuka]

In D106379#2891066, @dvyukov wrote:

"arc patch D106379" conflicts with origin/main

I hope this is fixed after I pushed all 3 commits.

Can you just set entire "parent/child" chain in Phabricator?

Where/how do I do this?

"Edit Related Revisions..." on the right menu of Phabricator. and check "Revision Contents" > Stack
I already linked D106350
"rename test Mutex to UserMutex" needs to be in there as well

[vitalybuka]

In D106379#2891094, @vitalybuka wrote:

In D106379#2891066, @dvyukov wrote:

"arc patch D106379" conflicts with origin/main

I hope this is fixed after I pushed all 3 commits.

Can you just set entire "parent/child" chain in Phabricator?

Where/how do I do this?

"Edit Related Revisions..." on the right menu of Phabricator. and check "Revision Contents" > Stack
I already linked D106350
"rename test Mutex to UserMutex" needs to be in there as well

D106379+D106350 do not conflict, but without "rename" patch i am not sure if it's in valid state.

[dvyukov]

In D106379#2891101, @vitalybuka wrote:

In D106379#2891094, @vitalybuka wrote:

In D106379#2891066, @dvyukov wrote:

"arc patch D106379" conflicts with origin/main

I hope this is fixed after I pushed all 3 commits.

Can you just set entire "parent/child" chain in Phabricator?

Where/how do I do this?

"Edit Related Revisions..." on the right menu of Phabricator. and check "Revision Contents" > Stack
I already linked D106350
"rename test Mutex to UserMutex" needs to be in there as well

D106379+D106350 do not conflict, but without "rename" patch i am not sure if it's in valid state.

This change is self-contained and applies on top of a recent main/HEAD:

$ git log --oneline
5d29184bb9bb (HEAD -> mutex-deadlock-detector) tsan: switch to the new sanitizer_common mutex
44e53dcd902c tsan: rename test Mutex to UserMutex
9d223a2c38f0 sanitizer_common: add deadlock detection to the Mutex2
15af3aaa2e8a (origin/main, main) [AArch64][SME] Add system registers and related instructions

[vitalybuka]

compiler-rt/lib/sanitizer_common/sanitizer_mutex.cpp

46 ↗

(On Diff #360203)

please add "static" here and below where appropriate

58 ↗

(On Diff #360203)

I guess "= {}" can insert interceptable memset

66 ↗

(On Diff #360203)

uptr and now (int) cast?

99 ↗

(On Diff #360203)

this could be nicely calculated with bit operations on stack with
u32 trans[kMutexTypeMax];

first we convert to:

u32 trans[kMutexTypeMax] = {};
  for (int i = 0; i < mutex_count; i++) {
    for (int j = 0; j < mutex_count; j++) 
      if (mutex_can_lock[i][j])
        trans[i] |= 1 << j;
  }
  for (int k = 0; k < mutex_count; k++) {
    for (int i = 0; i < mutex_count; i++) {
      if (trans[i] & (1 << k))
        # set bit j in trans[i] for each bit j in trans[k]
        for (int j = 0; j < mutex_count; j++) {
          if (trans[k] & (1 << j))
            trans[i] |= (1 << j);
        }
    }
  }

so

for (int k = 0; k < mutex_count; k++) {
    for (int i = 0; i < mutex_count; i++) {
      if (trans[i] & (1 << k))
        trans[i] |= trans[k];
    }
  }

but I didn't compile :)

198 ↗

(On Diff #360203)

to my taste CheckedMutex can be separated from "tsan: switch to the new sanitizer_common mutex"

compiler-rt/lib/sanitizer_common/sanitizer_mutex.h

152

What is the purpose of this inheritance?
Usually composition is recommended in cases like this, e.g. https://google.github.io/styleguide/cppguide.html#Inheritance

class MUTEX Mutex {
  CheckedMutex checked_;
 public:
  constexpr Mutex(MutexType type = MutexUnchecked) : checked_(type) {}

  void Lock() ACQUIRE() {
    checked_.Lock();
...

compiler-rt/lib/tsan/rtl/tsan_rtl_mutex.cpp

448 ↗

(On Diff #360203)

can you land boilerplate like this in a separate patches?

[vitalybuka]

compiler-rt/lib/sanitizer_common/sanitizer_mutex.cpp

66 ↗

(On Diff #360203)

uptr and now (int) cast?

Sorry, typos.
Avoid cast (int) by using uptr?

[dvyukov]

addressed comments

[dvyukov]

I've addressed all comments. PTAL. Now it's 4 commits:
https://github.com/dvyukov/llvm-project/pull/5/commits

compiler-rt/lib/sanitizer_common/sanitizer_mutex.cpp

46 ↗

(On Diff #360203)

Done.
Const objects are static by default (that's why consts can be declared in header files). But I added static to non-const variables below.

198 ↗

(On Diff #360203)

You mean separated into a separate commit? It's in a separate commit, this change contains 3, but did not find how to look at separate commits in Phabricator, so I uploaded this to github as well:
https://github.com/dvyukov/llvm-project/pull/5/commits

compiler-rt/lib/sanitizer_common/sanitizer_mutex.h

152

I used inheritance for the purposes of EBO:
https://en.cppreference.com/w/cpp/language/ebo
I would not like to make release Mutex larger than it could be.

[vitalybuka]

In D106379#2892563, @dvyukov wrote:

I've addressed all comments. PTAL. Now it's 4 commits:
https://github.com/dvyukov/llvm-project/pull/5/commits

I guess that's the feature you are looking for https://screenshot.googleplex.com/7N2V8vVmc8oKdmm

[vitalybuka]

In D106379#2894044, @vitalybuka wrote:

In D106379#2892563, @dvyukov wrote:

I've addressed all comments. PTAL. Now it's 4 commits:
https://github.com/dvyukov/llvm-project/pull/5/commits

I guess that's the feature you are looking for https://screenshot.googleplex.com/7N2V8vVmc8oKdmm

I mean if you upload them as a separate ones here, I'll stamp them and we move forward.

compiler-rt/lib/sanitizer_common/sanitizer_mutex.h

152

class MUTEX Mutex {
  [[no_unique_address]] CheckedMutex checked_;
 public:
  constexpr Mutex(MutexType type = MutexUnchecked) : checked_(type) {}

  void Lock() ACQUIRE() {
    checked_.Lock();

[vitalybuka]

In D106379#2894073, @vitalybuka wrote:

In D106379#2894044, @vitalybuka wrote:

In D106379#2892563, @dvyukov wrote:

I've addressed all comments. PTAL. Now it's 4 commits:
https://github.com/dvyukov/llvm-project/pull/5/commits

I guess that's the feature you are looking for https://screenshot.googleplex.com/7N2V8vVmc8oKdmm

I mean if you upload them as a separate ones here, I'll stamp them and we move forward.

And we can see them here https://screenshot.googleplex.com/brREoodQu2LLmzx

compiler-rt/lib/sanitizer_common/sanitizer_mutex.cpp

198 ↗

(On Diff #360203)

You mean separated into a separate commit? It's in a separate commit, this change contains 3, but did not find how to look at separate commits in Phabricator, so I uploaded this to github as well:
https://github.com/dvyukov/llvm-project/pull/5/commits

[vitalybuka]

compiler-rt/lib/tsan/rtl/tsan_defs.h

176

Something like this ?

enum class MutexType {
  Trace = MutexLastCommon,
  Report,
  SyncVar,
  Annotations,
  ...
};

compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp

437–439

Wasn't ScopedIgnoreInterceptors relevant for existing code? If so maybe a separate patch?

compiler-rt/lib/tsan/rtl/tsan_rtl.cpp

1135

is this leaked from another patch?

[vitalybuka]

After some poking around with arc tool I found this:

1. you can upload entire branch with: git rebase -i --exec 'arc diff HEAD^' origin/main

2. problem is that my "arc" didn't set parent/child relationship, and the reason is that it's not enabled on HEAD of arc. So you can set them manually in Phabricator or in arc diff message with "Depends on D????"

3. Automatic "Depends on" can be restored with

arcanist$ git checkout dc42f51^

Then you can:
git rebase -i --exec 'arc diff HEAD^' origin/main
and produce stack similar to pull request on github like here: https://reviews.llvm.org/D106492

[dvyukov]

I mean if you upload them as a separate ones here, I'll stamp them and we move forward.

You mean I could upload all commits as separate phabricator reviews, right?
arc help diff says it will always upload diff with the current HEAD, so I would need to checkout each commit one-by-one and do 'arc diff' for each of them, right? and then link them in the phabricator UI?

[dvyukov]

upload each commit into separate review

[dvyukov]

re-upload in the hope that arc will auto-set dependent reviews

[dvyukov]

reupload

[dvyukov]

OK, I hope I managed to upload and link everything properly. I've uploaded this as 6 separate changes linked together.
PTAL

compiler-rt/lib/tsan/rtl/tsan_defs.h

176

I like enum classes, but this will require casts for all uses.
We pass these to Mutex ctor which accepts the sanitizer_common enum (different type).
What we do here is a bit non-standard, we declare part of enum in sanitizer_common and then "extend" it in tsan. I don't know how to do it with enum classes.

compiler-rt/lib/tsan/rtl/tsan_rtl.cpp

1135

Right patch, but I am failing to upload this multi-commit change properly again. I did just 'arc diff' which uploaded only the last commit.

[vitalybuka]

I would need to checkout each commit one-by-one and do 'arc diff' for each of them, right? and then link them in the phabricator UI?

Did you see my comment with git rebase?

[dvyukov]

In D106379#2896944, @vitalybuka wrote:

I would need to checkout each commit one-by-one and do 'arc diff' for each of them, right? and then link them in the phabricator UI?

Did you see my comment with git rebase?

Yes, they answer my questions. I read them after writing this.

[melver]

compiler-rt/lib/tsan/rtl/tsan_rtl.cpp

1149

I was thinking we could add a static assert here that the size is below the max, but as long as there is a CHECK-fail in the deadlock detector code that the last element is the terminator, this is not needed.

compiler-rt/lib/sanitizer_common/sanitizer_mutex.h

#if SANITIZER_CHECK_DEADLOCKS
 
   void LockImpl(uptr pc);
   void UnlockImpl();
   static void CheckNoLocksImpl();
 #endif
 };
 
 // Reader-writer mutex.
-class MUTEX Mutex2 {
+class MUTEX Mutex {
  public:
-  constexpr Mutex2(MutexType type = MutexUnchecked) : checked_(type) {}
+  constexpr Mutex(MutexType type = MutexUnchecked) : checked_(type) {}
 
   void Lock() ACQUIRE() {
     checked_.Lock();
     u64 reset_mask = ~0ull;
     u64 state = atomic_load_relaxed(&state_);
     const uptr kMaxSpinIters = 1500;
     for (uptr spin_iters = 0;; spin_iters++) {
       u64 new_state;
static constexpr u64 kWaitingReaderMask = ((1ull << kCounterWidth) - 1)
                                             << kWaitingReaderShift;
   static constexpr u64 kWaitingWriterShift = 2 * kCounterWidth;
   static constexpr u64 kWaitingWriterInc = 1ull << kWaitingWriterShift;
   static constexpr u64 kWaitingWriterMask = ((1ull << kCounterWidth) - 1)
                                             << kWaitingWriterShift;
   static constexpr u64 kWriterLock = 1ull << (3 * kCounterWidth);
   static constexpr u64 kWriterSpinWait = 1ull << (3 * kCounterWidth + 1);
 
-  Mutex2(const Mutex2 &) = delete;
-  void operator=(const Mutex2 &) = delete;
+  Mutex(const Mutex &) = delete;
+  void operator=(const Mutex &) = delete;
 };
 
 void FutexWait(atomic_uint32_t *p, u32 cmp);
 void FutexWake(atomic_uint32_t *p, u32 count);
 
 class MUTEX BlockingMutex {
  public:
   explicit constexpr BlockingMutex(LinkerInitialized)
private:
   GenericScopedReadLock(const GenericScopedReadLock &) = delete;
   void operator=(const GenericScopedReadLock &) = delete;
 };
 
 typedef GenericScopedLock<StaticSpinMutex> SpinMutexLock;
 typedef GenericScopedLock<BlockingMutex> BlockingMutexLock;
 typedef GenericScopedLock<RWMutex> RWMutexLock;
 typedef GenericScopedReadLock<RWMutex> RWMutexReadLock;
+typedef GenericScopedLock<Mutex> Lock;
+typedef GenericScopedReadLock<Mutex> ReadLock;
 
 }  // namespace __sanitizer
 
 #endif  // SANITIZER_MUTEX_H

compiler-rt/lib/sanitizer_common/tests/sanitizer_mutex_test.cpp

TEST(SanitizerCommon, BlockingMutex) {
   pthread_t threads[kThreads];
   for (int i = 0; i < kThreads; i++)
     PTHREAD_CREATE(&threads[i], 0, lock_thread<BlockingMutex>, &data);
   for (int i = 0; i < kThreads; i++)
     PTHREAD_JOIN(threads[i], 0);
   check_locked(mtx);
 }
 
-TEST(SanitizerCommon, Mutex2) {
-  Mutex2 mtx;
-  TestData<Mutex2> data(&mtx);
+TEST(SanitizerCommon, Mutex) {
+  Mutex mtx;
+  TestData<Mutex> data(&mtx);
   pthread_t threads[kThreads];
   for (int i = 0; i < kThreads; i++)
-    PTHREAD_CREATE(&threads[i], 0, read_write_thread<Mutex2>, &data);
+    PTHREAD_CREATE(&threads[i], 0, read_write_thread<Mutex>, &data);
   for (int i = 0; i < kThreads; i++) PTHREAD_JOIN(threads[i], 0);
 }
 
 struct SemaphoreData {
   Semaphore *sem;
   bool done;
 };
 

compiler-rt/lib/tsan/CMakeLists.txt

set(TSAN_SOURCES
   rtl/tsan_interceptors_posix.cpp
   rtl/tsan_interface.cpp
   rtl/tsan_interface_ann.cpp
   rtl/tsan_interface_atomic.cpp
   rtl/tsan_interface_java.cpp
   rtl/tsan_malloc_mac.cpp
   rtl/tsan_md5.cpp
   rtl/tsan_mman.cpp
-  rtl/tsan_mutex.cpp
   rtl/tsan_mutexset.cpp
   rtl/tsan_preinit.cpp
   rtl/tsan_report.cpp
   rtl/tsan_rtl.cpp
   rtl/tsan_rtl_mutex.cpp
   rtl/tsan_rtl_proc.cpp
   rtl/tsan_rtl_report.cpp
   rtl/tsan_rtl_thread.cpp
set(TSAN_HEADERS
   rtl/tsan_flags.inc
   rtl/tsan_ignoreset.h
   rtl/tsan_interceptors.h
   rtl/tsan_interface.h
   rtl/tsan_interface_ann.h
   rtl/tsan_interface_inl.h
   rtl/tsan_interface_java.h
   rtl/tsan_mman.h
-  rtl/tsan_mutex.h
   rtl/tsan_mutexset.h
   rtl/tsan_platform.h
   rtl/tsan_ppc_regs.h
   rtl/tsan_report.h
   rtl/tsan_rtl.h
   rtl/tsan_stack_trace.h
   rtl/tsan_suppressions.h
   rtl/tsan_symbolize.h

compiler-rt/lib/tsan/go/build.bat

 type ^
   tsan_go.cpp ^
   ..\rtl\tsan_interface_atomic.cpp ^
   ..\rtl\tsan_clock.cpp ^
   ..\rtl\tsan_flags.cpp ^
   ..\rtl\tsan_md5.cpp ^
-  ..\rtl\tsan_mutex.cpp ^
   ..\rtl\tsan_report.cpp ^
   ..\rtl\tsan_rtl.cpp ^
   ..\rtl\tsan_rtl_mutex.cpp ^
   ..\rtl\tsan_rtl_report.cpp ^
   ..\rtl\tsan_rtl_thread.cpp ^
   ..\rtl\tsan_rtl_proc.cpp ^
   ..\rtl\tsan_suppressions.cpp ^
   ..\rtl\tsan_sync.cpp ^

compiler-rt/lib/tsan/go/buildgo.sh

 #!/bin/sh
 
 set -e
 
 SRCS="
   tsan_go.cpp
   ../rtl/tsan_clock.cpp
   ../rtl/tsan_external.cpp
   ../rtl/tsan_flags.cpp
   ../rtl/tsan_interface_atomic.cpp
   ../rtl/tsan_md5.cpp
-  ../rtl/tsan_mutex.cpp
   ../rtl/tsan_report.cpp
   ../rtl/tsan_rtl.cpp
   ../rtl/tsan_rtl_mutex.cpp
   ../rtl/tsan_rtl_report.cpp
   ../rtl/tsan_rtl_thread.cpp
   ../rtl/tsan_rtl_proc.cpp
   ../rtl/tsan_stack_trace.cpp
   ../rtl/tsan_suppressions.cpp

compiler-rt/lib/tsan/rtl/tsan_defs.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef TSAN_DEFS_H
 #define TSAN_DEFS_H
 
 #include "sanitizer_common/sanitizer_internal_defs.h"
 #include "sanitizer_common/sanitizer_libc.h"
+#include "sanitizer_common/sanitizer_mutex.h"
 #include "ubsan/ubsan_platform.h"
 
 // Setup defaults for compile definitions.
 #ifndef TSAN_NO_HISTORY
 # define TSAN_NO_HISTORY 0
 #endif
 
 #ifndef TSAN_CONTAINS_UBSAN
enum ExternalTag : uptr {
   kExternalTagNone = 0,
   kExternalTagSwiftModifyingAccess = 1,
   kExternalTagFirstUserAvailable = 2,
   kExternalTagMax = 1024,
   // Don't set kExternalTagMax over 65,536, since MBlock only stores tags
   // as 16-bit values, see tsan_defs.h.
 };
 
+enum MutexType {
+  MutexTypeTrace = MutexLastCommon,
+  MutexTypeReport,
+  MutexTypeSyncVar,
+  MutexTypeAnnotations,
+  MutexTypeAtExit,
+  MutexTypeFired,
+  MutexTypeRacy,
+  MutexTypeGlobalProc,
+};
+
 }  // namespace __tsan
 
 #endif  // TSAN_DEFS_H

compiler-rt/lib/tsan/rtl/tsan_dense_alloc.h

 // the index onto the real pointer. The index is u32, that is, 2 times smaller
 // than uptr (hense the Dense prefix).
 //===----------------------------------------------------------------------===//
 #ifndef TSAN_DENSE_ALLOC_H
 #define TSAN_DENSE_ALLOC_H
 
 #include "sanitizer_common/sanitizer_common.h"
 #include "tsan_defs.h"
-#include "tsan_mutex.h"
 
 namespace __tsan {
 
 class DenseSlabAllocCache {
   static const uptr kSize = 128;
   typedef u32 IndexT;
   uptr pos;
   IndexT cache[kSize];

compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp

 }
 
 ScopedInterceptor::~ScopedInterceptor() {
   if (!thr_->is_inited) return;
   DisableIgnores();
   if (!thr_->ignore_interceptors) {
     ProcessPendingSignals(thr_);
     FuncExit(thr_);
-    CheckNoLocks(thr_);
+    CheckedMutex::CheckNoLocks();
   }
 }
 
 void ScopedInterceptor::EnableIgnores() {
   if (ignoring_) {
     ThreadIgnoreBegin(thr_, pc_, /*save_stack=*/false);
     if (flags()->ignore_noninstrumented_modules) thr_->suppress_reports++;
     if (in_ignored_lib_) {
static int setup_at_exit_wrapper(ThreadState *thr, uptr pc, void(*f)(),
   ThreadIgnoreBegin(thr, pc);
   int res;
   if (!dso) {
     // NetBSD does not preserve the 2nd argument if dso is equal to 0
     // Store ctx in a local stack-like structure
 
     // Ensure thread-safety.
     BlockingMutexLock l(&interceptor_ctx()->atexit_mu);
     // __cxa_atexit calls calloc. If we don't ignore interceptors, we will fail
     // due to atexit_mu held on exit from the calloc interceptor.
     ScopedIgnoreInterceptors ignore;
 
     res = REAL(__cxa_atexit)((void (*)(void *a))at_exit_wrapper, 0, 0);
     // Push AtExitCtx on the top of the stack of callback functions
     if (!res) {
       interceptor_ctx()->AtExitStack.PushBack(ctx);
     }
   } else {
     res = REAL(__cxa_atexit)(cxa_at_exit_wrapper, ctx, dso);

compiler-rt/lib/tsan/rtl/tsan_interface_ann.cpp

 //
 //===----------------------------------------------------------------------===//
 #include "sanitizer_common/sanitizer_libc.h"
 #include "sanitizer_common/sanitizer_internal_defs.h"
 #include "sanitizer_common/sanitizer_placement_new.h"
 #include "sanitizer_common/sanitizer_stacktrace.h"
 #include "sanitizer_common/sanitizer_vector.h"
 #include "tsan_interface_ann.h"
-#include "tsan_mutex.h"
 #include "tsan_report.h"
 #include "tsan_rtl.h"
 #include "tsan_mman.h"
 #include "tsan_flags.h"
 #include "tsan_platform.h"
 
 #define CALLERPC ((uptr)__builtin_return_address(0))
 
 using namespace __tsan;
 
 namespace __tsan {
 
 class ScopedAnnotation {
  public:
   ScopedAnnotation(ThreadState *thr, const char *aname, uptr pc)
       : thr_(thr) {
     FuncEntry(thr_, pc);
     DPrintf("#%d: annotation %s()\n", thr_->tid, aname);
   }
 
   ~ScopedAnnotation() {
     FuncExit(thr_);
-    CheckNoLocks(thr_);
+    CheckedMutex::CheckNoLocks();
   }
  private:
   ThreadState *const thr_;
 };
 
 #define SCOPED_ANNOTATION_RET(typ, ret) \
     if (!flags()->enable_annotations) \
       return ret; \

compiler-rt/lib/tsan/rtl/tsan_interface_java.cpp

 //===-- tsan_interface_java.cpp -------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of ThreadSanitizer (TSan), a race detector.
 //
 //===----------------------------------------------------------------------===//
 
 #include "tsan_interface_java.h"
 #include "tsan_rtl.h"
-#include "tsan_mutex.h"
 #include "sanitizer_common/sanitizer_internal_defs.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_placement_new.h"
 #include "sanitizer_common/sanitizer_stacktrace.h"
 #include "sanitizer_common/sanitizer_procmaps.h"
 
 using namespace __tsan;
 

compiler-rt/lib/tsan/rtl/tsan_mutex.h

-//===-- tsan_mutex.h --------------------------------------------*- C++ -*-===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-//
-// This file is a part of ThreadSanitizer (TSan), a race detector.
-//
-//===----------------------------------------------------------------------===//
-#ifndef TSAN_MUTEX_H
-#define TSAN_MUTEX_H
-
-#include "sanitizer_common/sanitizer_atomic.h"
-#include "sanitizer_common/sanitizer_mutex.h"
-#include "tsan_defs.h"
-
-namespace __tsan {
-
-enum MutexType {
-  MutexTypeInvalid,
-  MutexTypeTrace,
-  MutexTypeThreads,
-  MutexTypeReport,
-  MutexTypeSyncVar,
-  MutexTypeSyncTab,
-  MutexTypeSlab,
-  MutexTypeAnnotations,
-  MutexTypeAtExit,
-  MutexTypeMBlock,
-  MutexTypeJavaMBlock,
-  MutexTypeDDetector,
-  MutexTypeFired,
-  MutexTypeRacy,
-  MutexTypeGlobalProc,
-
-  // This must be the last.
-  MutexTypeCount
-};
-
-class Mutex {
- public:
-  explicit Mutex(MutexType type);
-  ~Mutex();
-
-  void Lock();
-  void Unlock();
-
-  void ReadLock();
-  void ReadUnlock();
-
-  void CheckLocked();
-
- private:
-  atomic_uintptr_t state_;
-#if SANITIZER_DEBUG
-  MutexType type_;
-#endif
-
-  Mutex(const Mutex&);
-  void operator = (const Mutex&);
-};
-
-typedef GenericScopedLock<Mutex> Lock;
-typedef GenericScopedReadLock<Mutex> ReadLock;
-
-class InternalDeadlockDetector {
- public:
-  InternalDeadlockDetector();
-  void Lock(MutexType t);
-  void Unlock(MutexType t);
-  void CheckNoLocks();
- private:
-  u64 seq_;
-  u64 locked_[MutexTypeCount];
-};
-
-void InitializeMutex();
-
-// Checks that the current thread does not hold any runtime locks
-// (e.g. when returning from an interceptor).
-void CheckNoLocks(ThreadState *thr);
-
-}  // namespace __tsan
-
-#endif  // TSAN_MUTEX_H

compiler-rt/lib/tsan/rtl/tsan_mutex.cpp

-//===-- tsan_mutex.cpp ----------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-//
-// This file is a part of ThreadSanitizer (TSan), a race detector.
-//
-//===----------------------------------------------------------------------===//
-#include "sanitizer_common/sanitizer_libc.h"
-#include "tsan_mutex.h"
-#include "tsan_platform.h"
-#include "tsan_rtl.h"
-
-namespace __tsan {
-
-// Simple reader-writer spin-mutex. Optimized for not-so-contended case.
-// Readers have preference, can possibly starvate writers.
-
-// The table fixes what mutexes can be locked under what mutexes.
-// E.g. if the row for MutexTypeThreads contains MutexTypeReport,
-// then Report mutex can be locked while under Threads mutex.
-// The leaf mutexes can be locked under any other mutexes.
-// Recursive locking is not supported.
-#if SANITIZER_DEBUG && !SANITIZER_GO
-const MutexType MutexTypeLeaf = (MutexType)-1;
-static MutexType CanLockTab[MutexTypeCount][MutexTypeCount] = {
-  /*0  MutexTypeInvalid*/     {},
-  /*1  MutexTypeTrace*/       {MutexTypeLeaf},
-  /*2  MutexTypeThreads*/     {MutexTypeReport},
-  /*3  MutexTypeReport*/      {MutexTypeSyncVar,
-                               MutexTypeMBlock, MutexTypeJavaMBlock},
-  /*4  MutexTypeSyncVar*/     {MutexTypeDDetector},
-  /*5  MutexTypeSyncTab*/     {},  // unused
-  /*6  MutexTypeSlab*/        {MutexTypeLeaf},
-  /*7  MutexTypeAnnotations*/ {},
-  /*8  MutexTypeAtExit*/      {MutexTypeSyncVar},
-  /*9  MutexTypeMBlock*/      {MutexTypeSyncVar},
-  /*10 MutexTypeJavaMBlock*/  {MutexTypeSyncVar},
-  /*11 MutexTypeDDetector*/   {},
-  /*12 MutexTypeFired*/       {MutexTypeLeaf},
-  /*13 MutexTypeRacy*/        {MutexTypeLeaf},
-  /*14 MutexTypeGlobalProc*/  {},
-};
-
-static bool CanLockAdj[MutexTypeCount][MutexTypeCount];
-#endif
-
-void InitializeMutex() {
-#if SANITIZER_DEBUG && !SANITIZER_GO
-  // Build the "can lock" adjacency matrix.
-  // If [i][j]==true, then one can lock mutex j while under mutex i.
-  const int N = MutexTypeCount;
-  int cnt[N] = {};
-  bool leaf[N] = {};
-  for (int i = 1; i < N; i++) {
-    for (int j = 0; j < N; j++) {
-      MutexType z = CanLockTab[i][j];
-      if (z == MutexTypeInvalid)
-        continue;
-      if (z == MutexTypeLeaf) {
-        CHECK(!leaf[i]);
-        leaf[i] = true;
-        continue;
-      }
-      CHECK(!CanLockAdj[i][(int)z]);
-      CanLockAdj[i][(int)z] = true;
-      cnt[i]++;
-    }
-  }
-  for (int i = 0; i < N; i++) {
-    CHECK(!leaf[i] || cnt[i] == 0);
-  }
-  // Add leaf mutexes.
-  for (int i = 0; i < N; i++) {
-    if (!leaf[i])
-      continue;
-    for (int j = 0; j < N; j++) {
-      if (i == j || leaf[j] || j == MutexTypeInvalid)
-        continue;
-      CHECK(!CanLockAdj[j][i]);
-      CanLockAdj[j][i] = true;
-    }
-  }
-  // Build the transitive closure.
-  bool CanLockAdj2[MutexTypeCount][MutexTypeCount];
-  for (int i = 0; i < N; i++) {
-    for (int j = 0; j < N; j++) {
-      CanLockAdj2[i][j] = CanLockAdj[i][j];
-    }
-  }
-  for (int k = 0; k < N; k++) {
-    for (int i = 0; i < N; i++) {
-      for (int j = 0; j < N; j++) {
-        if (CanLockAdj2[i][k] && CanLockAdj2[k][j]) {
-          CanLockAdj2[i][j] = true;
-        }
-      }
-    }
-  }
-#if 0
-  Printf("Can lock graph:\n");
-  for (int i = 0; i < N; i++) {
-    for (int j = 0; j < N; j++) {
-      Printf("%d ", CanLockAdj[i][j]);
-    }
-    Printf("\n");
-  }
-  Printf("Can lock graph closure:\n");
-  for (int i = 0; i < N; i++) {
-    for (int j = 0; j < N; j++) {
-      Printf("%d ", CanLockAdj2[i][j]);
-    }
-    Printf("\n");
-  }
-#endif
-  // Verify that the graph is acyclic.
-  for (int i = 0; i < N; i++) {
-    if (CanLockAdj2[i][i]) {
-      Printf("Mutex %d participates in a cycle\n", i);
-      Die();
-    }
-  }
-#endif
-}
-
-InternalDeadlockDetector::InternalDeadlockDetector() {
-  // Rely on zero initialization because some mutexes can be locked before ctor.
-}
-
-#if SANITIZER_DEBUG && !SANITIZER_GO
-void InternalDeadlockDetector::Lock(MutexType t) {
-  // Printf("LOCK %d @%zu\n", t, seq_ + 1);
-  CHECK_GT(t, MutexTypeInvalid);
-  CHECK_LT(t, MutexTypeCount);
-  u64 max_seq = 0;
-  u64 max_idx = MutexTypeInvalid;
-  for (int i = 0; i != MutexTypeCount; i++) {
-    if (locked_[i] == 0)
-      continue;
-    CHECK_NE(locked_[i], max_seq);
-    if (max_seq < locked_[i]) {
-      max_seq = locked_[i];
-      max_idx = i;
-    }
-  }
-  locked_[t] = ++seq_;
-  if (max_idx == MutexTypeInvalid)
-    return;
-  // Printf("  last %d @%zu\n", max_idx, max_seq);
-  if (!CanLockAdj[max_idx][t]) {
-    Printf("ThreadSanitizer: internal deadlock detected\n");
-    Printf("ThreadSanitizer: can't lock %d while under %zu\n",
-               t, (uptr)max_idx);
-    CHECK(0);
-  }
-}
-
-void InternalDeadlockDetector::Unlock(MutexType t) {
-  // Printf("UNLO %d @%zu #%zu\n", t, seq_, locked_[t]);
-  CHECK(locked_[t]);
-  locked_[t] = 0;
-}
-
-void InternalDeadlockDetector::CheckNoLocks() {
-  for (int i = 0; i != MutexTypeCount; i++) {
-    CHECK_EQ(locked_[i], 0);
-  }
-}
-#endif
-
-void CheckNoLocks(ThreadState *thr) {
-#if SANITIZER_DEBUG && !SANITIZER_GO
-  thr->internal_deadlock_detector.CheckNoLocks();
-#endif
-}
-
-const uptr kUnlocked = 0;
-const uptr kWriteLock = 1;
-const uptr kReadLock = 2;
-
-class Backoff {
- public:
-  Backoff()
-    : iter_() {
-  }
-
-  bool Do() {
-    if (iter_++ < kActiveSpinIters)
-      proc_yield(kActiveSpinCnt);
-    else
-      internal_sched_yield();
-    return true;
-  }
-
-  u64 Contention() const {
-    u64 active = iter_ % kActiveSpinIters;
-    u64 passive = iter_ - active;
-    return active + 10 * passive;
-  }
-
- private:
-  int iter_;
-  static const int kActiveSpinIters = 10;
-  static const int kActiveSpinCnt = 20;
-};
-
-Mutex::Mutex(MutexType type) {
-  CHECK_GT(type, MutexTypeInvalid);
-  CHECK_LT(type, MutexTypeCount);
-#if SANITIZER_DEBUG
-  type_ = type;
-#endif
-  atomic_store(&state_, kUnlocked, memory_order_relaxed);
-}
-
-Mutex::~Mutex() {
-  CHECK_EQ(atomic_load(&state_, memory_order_relaxed), kUnlocked);
-}
-
-void Mutex::Lock() {
-#if SANITIZER_DEBUG && !SANITIZER_GO
-  cur_thread()->internal_deadlock_detector.Lock(type_);
-#endif
-  uptr cmp = kUnlocked;
-  if (atomic_compare_exchange_strong(&state_, &cmp, kWriteLock,
-                                     memory_order_acquire))
-    return;
-  for (Backoff backoff; backoff.Do();) {
-    if (atomic_load(&state_, memory_order_relaxed) == kUnlocked) {
-      cmp = kUnlocked;
-      if (atomic_compare_exchange_weak(&state_, &cmp, kWriteLock,
-                                       memory_order_acquire)) {
-        return;
-      }
-    }
-  }
-}
-
-void Mutex::Unlock() {
-  uptr prev = atomic_fetch_sub(&state_, kWriteLock, memory_order_release);
-  (void)prev;
-  DCHECK_NE(prev & kWriteLock, 0);
-#if SANITIZER_DEBUG && !SANITIZER_GO
-  cur_thread()->internal_deadlock_detector.Unlock(type_);
-#endif
-}
-
-void Mutex::ReadLock() {
-#if SANITIZER_DEBUG && !SANITIZER_GO
-  cur_thread()->internal_deadlock_detector.Lock(type_);
-#endif
-  uptr prev = atomic_fetch_add(&state_, kReadLock, memory_order_acquire);
-  if ((prev & kWriteLock) == 0)
-    return;
-  for (Backoff backoff; backoff.Do();) {
-    prev = atomic_load(&state_, memory_order_acquire);
-    if ((prev & kWriteLock) == 0) {
-      return;
-    }
-  }
-}
-
-void Mutex::ReadUnlock() {
-  uptr prev = atomic_fetch_sub(&state_, kReadLock, memory_order_release);
-  (void)prev;
-  DCHECK_EQ(prev & kWriteLock, 0);
-  DCHECK_GT(prev & ~kWriteLock, 0);
-#if SANITIZER_DEBUG && !SANITIZER_GO
-  cur_thread()->internal_deadlock_detector.Unlock(type_);
-#endif
-}
-
-void Mutex::CheckLocked() {
-  CHECK_NE(atomic_load(&state_, memory_order_relaxed), 0);
-}
-
-}  // namespace __tsan

compiler-rt/lib/tsan/rtl/tsan_rtl.cpp

 #endif
   if (common_flags()->detect_deadlocks)
     ctx->dd = DDetector::Create(flags());
   Processor *proc = ProcCreate();
   ProcWire(proc, thr);
   InitializeInterceptors();
   CheckShadowMapping();
   InitializePlatform();
-  InitializeMutex();
   InitializeDynamicAnnotations();
 #if !SANITIZER_GO
   InitializeShadowMemory();
   InitializeAllocatorLate();
   InstallDeadlySignalHandlers(TsanOnDeadlySignal);
 #endif
   // Setup correct file descriptor for error reports.
   __sanitizer_set_report_path(common_flags()->log_path);

 #if SANITIZER_DEBUG
 void build_consistency_debug() {}
 #else
 void build_consistency_release() {}
 #endif
 
 }  // namespace __tsan
 
+#if SANITIZER_CHECK_DEADLOCKS
+namespace __sanitizer {
+using namespace __tsan;
+MutexMeta mutex_meta[] = {
+    {MutexInvalid, "Invalid", {}},
+    {MutexThreadRegistry, "ThreadRegistry", {}},
+    {MutexTypeTrace, "Trace", {MutexLeaf}},
+    {MutexTypeReport, "Report", {MutexTypeSyncVar}},
+    {MutexTypeSyncVar, "SyncVar", {}},
+    {MutexTypeAnnotations, "Annotations", {}},
+    {MutexTypeAtExit, "AtExit", {MutexTypeSyncVar}},
+    {MutexTypeFired, "Fired", {MutexLeaf}},
+    {MutexTypeRacy, "Racy", {MutexLeaf}},
+    {MutexTypeGlobalProc, "GlobalProc", {}},
+    {},
+};
+
+void PrintMutexPC(uptr pc) { StackTrace(&pc, 1).Print(); }
+}  // namespace __sanitizer
+#endif
+
 #if !SANITIZER_GO
 // Must be included in this file to make sure everything is inlined.
-#include "tsan_interface_inl.h"
+#  include "tsan_interface_inl.h"
 #endif

compiler-rt/lib/tsan/rtl/tsan_rtl_report.cpp

static ReportStack *SymbolizeStack(StackTrace trace) {
   stack->frames = top;
   return stack;
 }
 
 bool ShouldReport(ThreadState *thr, ReportType typ) {
   // We set thr->suppress_reports in the fork context.
   // Taking any locking in the fork context can lead to deadlocks.
   // If any locks are already taken, it's too late to do this check.
-  CheckNoLocks(thr);
+  CheckedMutex::CheckNoLocks();
   // For the same reason check we didn't lock thread_registry yet.
   if (SANITIZER_DEBUG)
     ThreadRegistryLock l(ctx->thread_registry);
   if (!flags()->report_bugs || thr->suppress_reports)
     return false;
   switch (typ) {
     case ReportTypeSignalUnsafe:
       return flags()->report_signal_unsafe;
static bool RaceBetweenAtomicAndFree(ThreadState *thr) {
   if (s0.IsAtomic() && s1.IsFreed())
     return true;
   if (s1.IsAtomic() && thr->is_freeing)
     return true;
   return false;
 }
 
 void ReportRace(ThreadState *thr) {
-  CheckNoLocks(thr);
+  CheckedMutex::CheckNoLocks();
 
   // Symbolizer makes lots of intercepted calls. If we try to process them,
   // at best it will cause deadlocks on internal mutexes.
   ScopedIgnoreInterceptors ignore;
 
   if (!ShouldReport(thr, ReportTypeRace))
     return;
   if (!flags()->report_atomic_races && !RaceBetweenAtomicAndFree(thr))

compiler-rt/lib/tsan/rtl/tsan_sync.h

 #ifndef TSAN_SYNC_H
 #define TSAN_SYNC_H
 
 #include "sanitizer_common/sanitizer_atomic.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_deadlock_detector_interface.h"
 #include "tsan_defs.h"
 #include "tsan_clock.h"
-#include "tsan_mutex.h"
 #include "tsan_dense_alloc.h"
 
 namespace __tsan {
 
 // These need to match __tsan_mutex_* flags defined in tsan_interface.h.
 // See documentation there as well.
 enum MutexFlags {
   MutexFlagLinkerInit          = 1 << 0, // __tsan_mutex_linker_init

compiler-rt/lib/tsan/rtl/tsan_trace.h

 //===-- tsan_trace.h --------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of ThreadSanitizer (TSan), a race detector.
 //
 //===----------------------------------------------------------------------===//
 #ifndef TSAN_TRACE_H
 #define TSAN_TRACE_H
 
 #include "tsan_defs.h"
-#include "tsan_mutex.h"
 #include "tsan_stack_trace.h"
 #include "tsan_mutexset.h"
 
 namespace __tsan {
 
 const int kTracePartSizeBits = 13;
 const int kTracePartSize = 1 << kTracePartSizeBits;
 const int kTraceParts = 2 * 1024 * 1024 / kTracePartSize;

compiler-rt/lib/tsan/tests/unit/CMakeLists.txt

 set(TSAN_UNIT_TEST_SOURCES
   tsan_clock_test.cpp
   tsan_dense_alloc_test.cpp
   tsan_flags_test.cpp
   tsan_mman_test.cpp
-  tsan_mutex_test.cpp
   tsan_shadow_test.cpp
   tsan_stack_test.cpp
   tsan_sync_test.cpp
   tsan_unit_test_main.cpp
   )
 
 add_tsan_unittest(TsanUnitTest
   SOURCES ${TSAN_UNIT_TEST_SOURCES})

compiler-rt/lib/tsan/tests/unit/tsan_mutex_test.cpp

-//===-- tsan_mutex_test.cpp -----------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-//
-// This file is a part of ThreadSanitizer (TSan), a race detector.
-//
-//===----------------------------------------------------------------------===//
-#include "sanitizer_common/sanitizer_internal_defs.h"
-#include "sanitizer_common/sanitizer_atomic.h"
-#include "sanitizer_common/sanitizer_common.h"
-#include "sanitizer_common/sanitizer_mutex.h"
-#include "tsan_mutex.h"
-#include "gtest/gtest.h"
-
-namespace __tsan {
-
-template<typename MutexType>
-class TestData {
- public:
-  explicit TestData(MutexType *mtx)
-    : mtx_(mtx) {
-    for (int i = 0; i < kSize; i++)
-      data_[i] = 0;
-  }
-
-  void Write() {
-    Lock l(mtx_);
-    T v0 = data_[0];
-    for (int i = 0; i < kSize; i++) {
-      CHECK_EQ(data_[i], v0);
-      data_[i]++;
-    }
-  }
-
-  void Read() {
-    ReadLock l(mtx_);
-    T v0 = data_[0];
-    for (int i = 0; i < kSize; i++) {
-      CHECK_EQ(data_[i], v0);
-    }
-  }
-
-  void Backoff() {
-    volatile T data[kSize] = {};
-    for (int i = 0; i < kSize; i++) {
-      data[i]++;
-      CHECK_EQ(data[i], 1);
-    }
-  }
-
- private:
-  typedef GenericScopedLock<MutexType> Lock;
-  static const int kSize = 64;
-  typedef u64 T;
-  MutexType *mtx_;
-  char pad_[kCacheLineSize];
-  T data_[kSize];
-};
-
-const int kThreads = 8;
-const int kWriteRate = 1024;
-#if SANITIZER_DEBUG
-const int kIters = 16*1024;
-#else
-const int kIters = 64*1024;
-#endif
-
-template<typename MutexType>
-static void *write_mutex_thread(void *param) {
-  TestData<MutexType> *data = (TestData<MutexType>*)param;
-  for (int i = 0; i < kIters; i++) {
-    data->Write();
-    data->Backoff();
-  }
-  return 0;
-}
-
-template<typename MutexType>
-static void *read_mutex_thread(void *param) {
-  TestData<MutexType> *data = (TestData<MutexType>*)param;
-  for (int i = 0; i < kIters; i++) {
-    if ((i % kWriteRate) == 0)
-      data->Write();
-    else
-      data->Read();
-    data->Backoff();
-  }
-  return 0;
-}
-
-TEST(Mutex, Write) {
-  Mutex mtx(MutexTypeAnnotations);
-  TestData<Mutex> data(&mtx);
-  pthread_t threads[kThreads];
-  for (int i = 0; i < kThreads; i++)
-    pthread_create(&threads[i], 0, write_mutex_thread<Mutex>, &data);
-  for (int i = 0; i < kThreads; i++)
-    pthread_join(threads[i], 0);
-}
-
-TEST(Mutex, ReadWrite) {
-  Mutex mtx(MutexTypeAnnotations);
-  TestData<Mutex> data(&mtx);
-  pthread_t threads[kThreads];
-  for (int i = 0; i < kThreads; i++)
-    pthread_create(&threads[i], 0, read_mutex_thread<Mutex>, &data);
-  for (int i = 0; i < kThreads; i++)
-    pthread_join(threads[i], 0);
-}
-
-TEST(Mutex, SpinWrite) {
-  SpinMutex mtx;
-  TestData<SpinMutex> data(&mtx);
-  pthread_t threads[kThreads];
-  for (int i = 0; i < kThreads; i++)
-    pthread_create(&threads[i], 0, write_mutex_thread<SpinMutex>, &data);
-  for (int i = 0; i < kThreads; i++)
-    pthread_join(threads[i], 0);
-}
-
-}  // namespace __tsan