This is an archive of the discontinued LLVM Phabricator instance.

[scudo] Add the thread-safety annotations
ClosedPublic

Authored by Chia-hungDuan on Dec 27 2022, 1:21 PM.

Details

Summary

This CL adds the proper thread-safety annotations for most of the
functions and variables. However, given the restriction of the current
architecture, in some cases, we may not be able to use the annotations
easily. The followings are two exceptions,

  1. enable()/disable(): Many structures in scudo are enabled/disabled by acquiring the lock in each instance. This makes those structure act like a lock. We can't mark those functions with ACQUIRE()/RELEASE() because that makes the entire allocator become another lock. In the end, that implies we need to *acquire* the allocator before each malloc et al. request. Therefore, adding a variable to tell the status of those structures may be a better way to cooperate with thread-safety annotation.
  1. TSD/TSD shared/TSD exclusive: These three have simiar restrictions as mentioned above. In addition, they don't always need to be released if it's a thread local instance. However, thread-safety analysis doesn't support conditional branch. Which means we can't mark the proper annotations around the uses of TSDs. We may consider to make it consistent and which makes the code structure simpler.

This CL is supposed to introduce the annotations with the least code
refactoring. So only trivial thread safety issues will be addressed
here. For example, lacking of acquiring certain lock before accessing
certain variables will have the ScopedLock inserted. Other than that,
they are supposed to be done in the later changes.

Diff Detail

Event Timeline

Chia-hungDuan created this revision.Dec 27 2022, 1:21 PM
Herald added a project: Restricted Project. · View Herald TranscriptDec 27 2022, 1:21 PM
Chia-hungDuan requested review of this revision.Dec 27 2022, 1:21 PM
Herald added a project: Restricted Project. · View Herald TranscriptDec 27 2022, 1:21 PM
Herald added a subscriber: Restricted Project. · View Herald Transcript

This CL is the preparation for later lock granularity improvement and for now, it's supposed to have as small impact as possible. The only thing I'm considering to fix here is accessing PossibleRegions in SizeClassAllocator32. It seems to me that we don't have a dedicated lock on it (however, haven't seen issues though). I'll confirm the case and if it's critical, I may include it as well.

BTW, I'll add the reviewers in the next year (few days away :)). Now it's just a heads-up.

Added few more annotations

High level question: was this tested with Fuchsia/Linux/Android?

High level question: was this tested with Fuchsia/Linux/Android?

This has been tested on Linux and Android. Will seek the help from Fuchsia folks to verify the CL (And maybe they want to implement the HybridMutex::assertHeld())

Fix a warning on Android

@supersymetrie , Could you help check if there's any build issue on Fuchsia? Thanks

vitalybuka added inline comments.
compiler-rt/lib/scudo/standalone/CMakeLists.txt
34

It should be similar to COMPILER_RT_HAS_FNO_LTO_FLAG

compiler-rt/lib/scudo/standalone/combined.h
254

please split into smaller patches, where each patch annotate particular component

990

Please extract {} into a separate patch, or rather revert
This code uses no {} for single statements ifs

Chia-hungDuan added inline comments.Jan 4 2023, 4:48 PM
compiler-rt/lib/scudo/standalone/combined.h
254

Once we tag the mutex(HybridMutex) with proper capability, all the users will need to mark the proper annotations and we may end up touching the similar amount of files like this.

I try to keep the code structure as close before as possible so that it makes the review easier. (Those more complicated cases mentioned in the commit message are simply marked as NO_THREAD_SAFETY_ANALYSIS now and will be addressed later. )

I know it may still take some time on reviewing this but I hope this can be not that painful.

990

To clarify this, I suppose this is following LLVM coding-style. Even though from the official guideline, there's no specific case for this one. This is single statement in multiple lines and I usually see people add brackets in this case. I tend to follow this style here let me know if there's a different convention in Scudo.

On the other hand, I think this is simple style nit while I review the thread annotation of each function so I just sneak them in the change. I thought it's not too bad to fix them here but if you think it's still better to have them separately, I will do it.

cferris added inline comments.Jan 12 2023, 3:25 PM
compiler-rt/lib/scudo/standalone/combined.h
254

After looking at this change, I think it might make sense to break this up into two changes.

The first change includes the few brace additions and the changes to how the locks are being acquired/released but without any of the thread annotations. Specifically, including the changes to the locking around getStats. That is a little bit complicated, and I think it would help to see that in a single change.

compiler-rt/lib/scudo/standalone/mutex.h
56–60

I think it might be better to move this into the assertHeldImpl implementation. I say that because it is not obvious if you find the definition that this is linux only. But if it's actually in the implementation, there is no way to miss it's not doing anything except on linux.

compiler-rt/lib/scudo/standalone/primary32.h
212

disabling

213

locking

compiler-rt/lib/scudo/standalone/primary64.h
235

disabling

236

locking

319–320

This makes me a little nervous since the function is actually slightly lying. Unfortunately, this goes back to an already established API, so there is not way to change it from const char*. It probably should have been void* in the beginning, but too late now.

356

There doesn't appear to be a comment a few lines above. Is this still relevant?

394

Remove "the", before thread creation.

644

This is really helpful for tracking issues. Would it be possible to pass in the RegionInfoPtr to the function and not lock it?

Something like:

void getStats(ScopedString *Str, const RegionInfo *LockedRegion = nullptr)

You would have to use something other than a scoped lock though.

compiler-rt/lib/scudo/standalone/thread_annotations.h
14

I think ignored might be a better description of the action taken.

compiler-rt/lib/scudo/standalone/tsd_exclusive.h
66–72

removing

67

locking

68

properly mark

compiler-rt/lib/scudo/standalone/tsd_shared.h
86

which

165–168

which

Chia-hungDuan added inline comments.Jan 12 2023, 4:03 PM
compiler-rt/lib/scudo/standalone/combined.h
254

Got it and agree. I'll split out the linting fixes to a different one.

Chia-hungDuan marked 12 inline comments as done.

Address review comments

Chia-hungDuan added inline comments.Jan 18 2023, 3:36 PM
compiler-rt/lib/scudo/standalone/CMakeLists.txt
34

Sorry I checked the uses of COMPILER_RT_HAS_FNO_LTO_FLAG but not sure what do you mean. Can you share more details?

compiler-rt/lib/scudo/standalone/primary64.h
319–320

Agree. I was hesitating if we should do this casting. (We kind of do an UB above another UB...)

I decide to mark it as NO_THREAD_SAFETY_ANALYSIS and add a TODO now. Will fix it in the later patch

356

A left over. Removed.

398–400

Will fix this weird indentation by moving the comments up to the variables.

644

I was thinking to move this to its caller like

TransferBatch *popBatch(CacheT *C, uptr ClassId) {
  ..
  bool RegionExhausted = false;
  {
    ScopedLock L(Region->Mutex);
    ...
    if (!populateFreeList(...)) {
      ..
      RegionExhausted = Region->Exhausted;
    }
  }
  if (RegionExhausted)
    getStats()

In order to make as less impact as possible in the code structure in this CL. I'll have another patch to bring this back and ensure they get merged consecutively.

vitalybuka added inline comments.
compiler-rt/lib/scudo/standalone/CMakeLists.txt
34

Please rebase onto 0ce4fca3168e or later and use COMPILER_RT_HAS_WTHREAD_SAFETY_FLAG
I guess it's safe to assume support of -Werror=thread-safety in this case

compiler-rt/lib/scudo/standalone/combined.h
254

it's not just review, helps with bisecting regressions
even it's suppose to be compile time only change, it may accidentally incorporate functional change.

can be this done instead top to bottom?

730–731

this does not look like annotation change, would be nice to have separately with explanation

990

The convention is consistency, from the llvm standard:

As with other coding standards, individual projects, such as the Clang Static Analyzer, may have preexisting styles that do not conform to this. If a different formatting scheme is used consistently throughout the project, use that style instead. Otherwise, this standard applies to all LLVM tools, including clang, clang-tidy, and so on.

1171

Please introduce geters in a separate patch(es)

Chia-hungDuan added inline comments.Jan 18 2023, 7:47 PM
compiler-rt/lib/scudo/standalone/combined.h
254

I misunderstood this comment. Will do further split

990

I'm not saying this is not consistency. In fact, this kind of case is not consistent in many projects of LLVM, even in Scudo. I was trying to make it consistent. Anyway, linting fixes have been excluded.

hctim added a comment.Jan 19 2023, 9:24 AM

Thanks for the patch, I think this is a really good addition to scudo. Clearly you've found some bugs. An overall comment, there's a lot of mixed non-functional and functional changes here. For example, it's my understanding that this patch is primarily to add thread safety annotations, but there's functional changes in unmapTestOnly, malloc_info, and other places.

Can you please split the patch up into:

  1. Functional changes, one patch per fix.
  2. Annotations and any related small refactorings, one class per patch.
  3. A final patch to turn on the warnings.

Address review comment

Chia-hungDuan marked 3 inline comments as done.Jan 19 2023, 11:25 AM

Thanks for the patch, I think this is a really good addition to scudo. Clearly you've found some bugs. An overall comment, there's a lot of mixed non-functional and functional changes here. For example, it's my understanding that this patch is primarily to add thread safety annotations, but there's functional changes in unmapTestOnly, malloc_info, and other places.

Can you please split the patch up into:

  1. Functional changes, one patch per fix.
  2. Annotations and any related small refactorings, one class per patch.
  3. A final patch to turn on the warnings.

Thanks for the review!

Now it's split into 3 (4 if we include one additional slightly related fix),
Bug fix for getStats() is in https://reviews.llvm.org/D142149 (A related fix of this one is https://reviews.llvm.org/D141955)
This CL includes only adding annotations
The getter os TSD is in https://reviews.llvm.org/D142151

Chia-hungDuan updated this revision to Diff 490655.EditedJan 19 2023, 2:01 PM

Split out iterateOverChunks fix in D142157

cferris accepted this revision.Feb 2 2023, 3:58 PM

LGTM

This revision is now accepted and ready to land.Feb 2 2023, 3:58 PM
This revision was automatically updated to reflect the committed changes.