dvyukov (Dmitry Vyukov)
User

Projects

User does not belong to any projects.

User Details

User Since
Dec 6 2012, 2:31 AM (296 w, 6 d)

Recent Activity

Mon, Aug 13

dvyukov added inline comments to D50549: [libcxx] [test] Repair thread unsafety in thread tests.
Mon, Aug 13, 4:41 PM
dvyukov added inline comments to D50549: [libcxx] [test] Repair thread unsafety in thread tests.
Mon, Aug 13, 10:26 AM
dvyukov added inline comments to D50549: [libcxx] [test] Repair thread unsafety in thread tests.
Mon, Aug 13, 10:07 AM
dvyukov added inline comments to D50549: [libcxx] [test] Repair thread unsafety in thread tests.
Mon, Aug 13, 9:56 AM

Thu, Jul 26

dvyukov committed rL338023: [tsan] Fix gcc pedantic warning.
[tsan] Fix gcc pedantic warning
Thu, Jul 26, 6:03 AM
dvyukov committed rCRT338023: [tsan] Fix gcc pedantic warning.
[tsan] Fix gcc pedantic warning
Thu, Jul 26, 6:03 AM
dvyukov closed D49817: [tsan] Fix gcc pedantic warning.

Sure. Merged as 338023. Thanks.

Thu, Jul 26, 6:03 AM

Wed, Jul 25

dvyukov accepted D49817: [tsan] Fix gcc pedantic warning.
Wed, Jul 25, 10:56 PM

Tue, Jul 24

dvyukov accepted D49707: [tsan] Fix crash in objc_sync_enter/objc_sync_exit when using an Obj-C tagged pointer.
Tue, Jul 24, 12:49 AM · Restricted Project

Fri, Jul 20

dvyukov committed rCRT337550: esan: fix shadow setup.
esan: fix shadow setup
Fri, Jul 20, 6:45 AM
dvyukov committed rL337550: esan: fix shadow setup.
esan: fix shadow setup
Fri, Jul 20, 6:45 AM
dvyukov closed D49367: sanitizers: consistently check result of MmapFixedNoReserve.

Submitted in 337531

Fri, Jul 20, 1:40 AM
dvyukov added inline comments to D49367: sanitizers: consistently check result of MmapFixedNoReserve.
Fri, Jul 20, 1:40 AM
dvyukov committed rCRT337531: sanitizers: consistently check result of MmapFixedNoReserve.
sanitizers: consistently check result of MmapFixedNoReserve
Fri, Jul 20, 1:39 AM
dvyukov committed rL337531: sanitizers: consistently check result of MmapFixedNoReserve.
sanitizers: consistently check result of MmapFixedNoReserve
Fri, Jul 20, 1:38 AM

Jul 16 2018

dvyukov created D49367: sanitizers: consistently check result of MmapFixedNoReserve.
Jul 16 2018, 2:56 AM

Jun 22 2018

dvyukov committed rCRT335322: tsan: fix deficiency in MutexReadOrWriteUnlock.
tsan: fix deficiency in MutexReadOrWriteUnlock
Jun 22 2018, 1:32 AM
dvyukov committed rL335322: tsan: fix deficiency in MutexReadOrWriteUnlock.
tsan: fix deficiency in MutexReadOrWriteUnlock
Jun 22 2018, 1:32 AM

Jun 14 2018

dvyukov added a comment to D32915: [msan] Fix PR32842.

Nice!

Jun 14 2018, 10:40 PM · Restricted Project

Jun 13 2018

dvyukov accepted D48097: [TSan] Fix madvise(MADV_NOHUGEPAGE) for meta shadow memory.
Jun 13 2018, 8:03 AM

Jun 12 2018

dvyukov added inline comments to D48097: [TSan] Fix madvise(MADV_NOHUGEPAGE) for meta shadow memory.
Jun 12 2018, 10:50 PM

Jun 9 2018

dvyukov added a comment to D47977: [Fuzzer] First step to thread affinity.

And we did not even get to windows, mac, fuchsia and akaros.

Jun 9 2018, 3:33 AM
dvyukov added a comment to D47977: [Fuzzer] First step to thread affinity.

Well on Linux you can check process infos on /proc/<proc>/status I believe but that might be overkill.

Jun 9 2018, 3:30 AM
dvyukov added a comment to D47977: [Fuzzer] First step to thread affinity.

True ideally what I tried to say in the description is ideally the cpu load ought to be checked first but that s a lot of code :-)

Jun 9 2018, 3:18 AM
dvyukov added a comment to D47977: [Fuzzer] First step to thread affinity.

I understand your points, but usually see the jobs are often bound to cpu0 while this cpu can be potentially pretty busy.

Jun 9 2018, 2:54 AM
dvyukov added a comment to D47977: [Fuzzer] First step to thread affinity.

What's the gain? Performance improvement? What are the numbers for benchmarking?

Jun 9 2018, 2:36 AM

Jun 7 2018

dvyukov accepted D47289: [scudo] Improve the scalability of the shared TSD model.

So your current version is actually the best one.
Thanks for bearing with me.

Jun 7 2018, 10:47 PM
dvyukov added a comment to D47289: [scudo] Improve the scalability of the shared TSD model.

What exactly versions did you test? Current vs removing precedence and trylocking all caches? If yes, then it's unclear what factor affects performance as we change 2 things at the same time. So I would also try:

  1. Remove precedence, but scan only 4 random caches, then lock the current cache.
  2. Remove precedence, but scan only 4 random caches, then lock a random cache.
  3. Keep precedence but scan all caches.

I would not expect 3 to be faster, but who knows.
Looking at your results I think it mostly needs to be tested on t-test1, because it should not affect android with 2/4 caches.

Jun 7 2018, 11:30 AM
dvyukov added a comment to D47107: Vector clock algorithm improvement in case of blocking release sequence.

Hi Yorov,

Jun 7 2018, 7:50 AM

Jun 3 2018

dvyukov accepted D47677: [TSan] FreeBSD / intercept thr_exit.
Jun 3 2018, 10:54 PM

May 26 2018

dvyukov added inline comments to D47289: [scudo] Improve the scalability of the shared TSD model.
May 26 2018, 1:28 AM

May 16 2018

dvyukov accepted D46961: [scudo] Quarantine optimization.

I see. Let's wait for C++14.

May 16 2018, 10:59 AM
dvyukov added a comment to D46961: [scudo] Quarantine optimization.

Making it static const makes the compiler use cxa_guard_acquire & cxa_guard_release which we don't want to be able to stay free of libc++ dependencies.

May 16 2018, 10:45 AM
dvyukov added a comment to D46961: [scudo] Quarantine optimization.
  • const uptr BatchClassId = SizeClassMap::ClassID(sizeof(QuarantineBatch));
May 16 2018, 10:22 AM

May 13 2018

dvyukov accepted D46793: [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock.
May 13 2018, 1:38 AM

May 11 2018

dvyukov added a comment to D46793: [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock.

Interesting. Who calls these functions? Why? Can we change them to call normal functions instead?

May 11 2018, 11:57 PM

May 9 2018

dvyukov accepted D46597: [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms.
May 9 2018, 9:10 AM
dvyukov added inline comments to D46597: [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms.
May 9 2018, 1:07 AM

May 7 2018

dvyukov added a comment to D44881: [sanitizer_common] Ignore unloading of suppressed library.

There are several reasons:

  1. The address range is still ignored after unload. So if anything else loaded at that range it will be falsely ignore and this can lead to both false positives and false negatives.
  2. The list of ignored libraries is append-only, so it's not possible to remove from it.
  3. If a library is unloaded, it can be loaded again later. This will lead to unbounded growth of the list of ignored libraries. The list is fixed size, so soon we will crash with an obscure CHECK failure.
May 7 2018, 11:38 AM

Apr 30 2018

dvyukov committed rCRT331163: tsan: disable trace switching after multithreaded fork.
tsan: disable trace switching after multithreaded fork
Apr 30 2018, 12:33 AM
dvyukov committed rL331163: tsan: disable trace switching after multithreaded fork.
tsan: disable trace switching after multithreaded fork
Apr 30 2018, 12:32 AM

Apr 27 2018

dvyukov committed rCRT331023: tsan: improve "destroy of a locked mutex" reports.
tsan: improve "destroy of a locked mutex" reports
Apr 27 2018, 2:03 AM
dvyukov committed rL331023: tsan: improve "destroy of a locked mutex" reports.
tsan: improve "destroy of a locked mutex" reports
Apr 27 2018, 2:03 AM

Apr 23 2018

dvyukov added a comment to D45646: [tsan] Zero out the shadow memory for the stack and TLS in ThreadFinish.

I'm not too worried about the first variation. It's very unlikely that user code will try to mmap(MAP_FIXED)

Apr 23 2018, 2:47 AM · Restricted Project

Apr 20 2018

dvyukov added a comment to D45838: [sanitizer] More dead code removal.

Then I don't understand what happens here anymore. Go and drop it.

Apr 20 2018, 10:02 AM
dvyukov added a comment to D45838: [sanitizer] More dead code removal.

Tests are not linked with interceptors?

I didn't find anything that suggested that this particular one was, but someone else can probably chime in on that.
sanitizer::SanitizerSetThreadName is present in other binaries (Asan-x86-64-calls-Tests for example) and calls interceptor_prctl there, but is not called by anything.

Apr 20 2018, 9:27 AM
dvyukov added a comment to D45838: [sanitizer] More dead code removal.

Tests are not linked with interceptors?

Apr 20 2018, 9:12 AM
dvyukov added a comment to D45838: [sanitizer] More dead code removal.

I am not following. Isn't it a test for prctl interceptor? Why are we removing useful tests?
Perhaps it's not the most critical functionality. And perhaps SanitizerSetThreadName needs to be moved to the test itself. But still it's a test.

Apr 20 2018, 12:55 AM

Apr 19 2018

dvyukov added a comment to D45646: [tsan] Zero out the shadow memory for the stack and TLS in ThreadFinish.

Let's concentrate on correctness for now.
Shadow memory is not guaranteed to be zeros for new memory. At munmap does not clear it, and probably some other things. Perhaps if you use munmap in test, it will be easier to catch this (e.g. mmap a large chunk of memory, write to it, munmap, create a thread).

Apr 19 2018, 2:10 AM · Restricted Project
dvyukov committed rCRT330312: tsan: fix compiler warnings.
tsan: fix compiler warnings
Apr 19 2018, 12:46 AM
dvyukov committed rL330312: tsan: fix compiler warnings.
tsan: fix compiler warnings
Apr 19 2018, 12:46 AM

Apr 16 2018

dvyukov committed rCRT330122: tsan: add support for linux/powerpc64 in buildgo.sh.
tsan: add support for linux/powerpc64 in buildgo.sh
Apr 16 2018, 4:51 AM
dvyukov committed rL330122: tsan: add support for linux/powerpc64 in buildgo.sh.
tsan: add support for linux/powerpc64 in buildgo.sh
Apr 16 2018, 4:47 AM
dvyukov closed D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Submitted in:
http://llvm.org/viewvc/llvm-project?view=revision&revision=330122
Thanks!

Apr 16 2018, 4:47 AM
dvyukov added a comment to D45646: [tsan] Zero out the shadow memory for the stack and TLS in ThreadFinish.

Two things that I don't like here:

  1. This imposes cost of zeroing of up to 32MB (standard 8MB stack x 4x shadow) per thread creation/destruction for all OSes. Some programs create threads like insane.
  2. I don't think this fixes the actual root cause, only makes it even harder to localize. Note that cur_thread_finalize already clears the shadow slot, so if pthread reuses stack/tls wholesale, then the slot should be zero already. However, tsan does not generally keep shadow clear (e.g. munmap does not clear shadow too, and most likely a bunch of other things). So if the slot reuses memory from a previous mmap, it will crash the same way.

I wonder if moving the slot to _meta_ shadow is the right things to do. We actually clear meta shadow on unmap. I don't see where we clear stack, but we should, otherwise we can leak lots of sync objects on stack.

Apr 16 2018, 2:09 AM · Restricted Project

Apr 13 2018

dvyukov accepted D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Looks good to me.
Do you have commit access, or you want me to commit it?

Apr 13 2018, 8:56 AM
dvyukov added a comment to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Much nicer now!
I think this is ready for commit after fixing these few nits.

Apr 13 2018, 12:56 AM

Apr 12 2018

dvyukov added inline comments to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.
Apr 12 2018, 3:11 AM

Apr 11 2018

dvyukov added a reviewer for D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh: dvyukov.
Apr 11 2018, 1:52 AM
dvyukov added inline comments to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.
Apr 11 2018, 1:51 AM

Apr 5 2018

dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

No, it explicitly checks alignment to decide whether to make a libcall itself.

Apr 5 2018, 9:41 AM
dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

I've just checked the LangRef and realised this is false. LLVM does make such loads & stores UB, though I still think the Clang use is valid (if unfortunate).

Apr 5 2018, 8:01 AM
dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

Atomics accessed via C11 _Atomic and C++11 std::atomic will be suitably aligned, but there's a reasonable amount of legacy code that uses the GCC builtins on non-atomic types (of unknown alignment) and this is what Clang uses to implement those accesses when they come up.

Apr 5 2018, 7:31 AM
dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

How how can atomic objects end up being mis-aligned? Isn't it UB already?

Apr 5 2018, 6:55 AM

Apr 3 2018

dvyukov added a comment to D44981: asan: kernel: make no_sanitize("address") attribute work with -fsanitize=kernel-address.

You are right, it's not the best idea.
Is it too late to get rid of C/C++-level "kernel-address" attribute entirely? For the source code perspective, it's just ASan, but for the kernel. If that's an important distinction, one can always do #ifdef KERNEL or something.

Apr 3 2018, 1:11 PM

Mar 21 2018

dvyukov added a comment to D44714: Change TSAN external symbolization API to support returning multiple frames.

This broke darwin build. I've fixed it up in:
http://llvm.org/viewvc/llvm-project?view=revision&revision=328082
Both external symbolization tests still pass, so I hope it works for your use case.

Mar 21 2018, 2:30 AM
dvyukov committed rL328082: tsan: fix darwin build after 328079.
tsan: fix darwin build after 328079
Mar 21 2018, 2:29 AM
dvyukov committed rCRT328082: tsan: fix darwin build after 328079.
tsan: fix darwin build after 328079
Mar 21 2018, 2:29 AM
dvyukov added a comment to D44714: Change TSAN external symbolization API to support returning multiple frames.

Submitted in rev 328079.

Mar 21 2018, 1:47 AM
dvyukov committed rL328079: tsan: support inlined frames in external symbolization.
tsan: support inlined frames in external symbolization
Mar 21 2018, 1:47 AM
dvyukov committed rCRT328079: tsan: support inlined frames in external symbolization.
tsan: support inlined frames in external symbolization
Mar 21 2018, 1:47 AM
dvyukov accepted D44714: Change TSAN external symbolization API to support returning multiple frames.

Looks good. Comitting.

Mar 21 2018, 1:43 AM

Mar 16 2018

dvyukov committed rL327700: tsan: revert: Update buildgo.sh to pass -isysroot on Darwin..
tsan: revert: Update buildgo.sh to pass -isysroot on Darwin.
Mar 16 2018, 3:23 AM
dvyukov committed rCRT327700: tsan: revert: Update buildgo.sh to pass -isysroot on Darwin..
tsan: revert: Update buildgo.sh to pass -isysroot on Darwin.
Mar 16 2018, 3:23 AM

Mar 15 2018

dvyukov accepted D44071: [TSan] fix Go runtime test on amd64 with PIE.

LGTM
It actually fails on my laptop in the same way now. But works with this patch.

Mar 15 2018, 3:20 AM

Mar 7 2018

dvyukov accepted D44246: [sanitizer] Generalize atomic_uint8_t, atomic_uint16_t, ... into a template. NFC..
Mar 7 2018, 11:01 PM · Restricted Project

Feb 14 2018

dvyukov accepted D43325: [TSan] Fix static TLS boundaries calculations in __tls_get_addr interceptor..
Feb 14 2018, 10:40 PM

Feb 7 2018

dvyukov added a comment to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Is there an upstream Go bug for race on powerpc? Please CC me on it.

Feb 7 2018, 10:16 AM

Jan 29 2018

dvyukov committed rL323657: tsan: deflake a test.
tsan: deflake a test
Jan 29 2018, 7:12 AM
dvyukov closed D42633: tsan: deflake a test.

Submitted in r323657.

Jan 29 2018, 7:12 AM
dvyukov committed rCRT323657: tsan: deflake a test.
tsan: deflake a test
Jan 29 2018, 7:11 AM
dvyukov created D42633: tsan: deflake a test.
Jan 29 2018, 12:45 AM

Jan 22 2018

dvyukov closed D42384: asan: allow inline instrumentation for the kernel.

Committed in r323140.

Jan 22 2018, 11:08 AM
dvyukov committed rL323140: asan: allow inline instrumentation for the kernel.
asan: allow inline instrumentation for the kernel
Jan 22 2018, 11:08 AM
dvyukov accepted D42384: asan: allow inline instrumentation for the kernel.
Jan 22 2018, 10:40 AM

Dec 13 2017

dvyukov accepted D41190: [tsan] Separate the constants in libignore and bump the maximum for instrumented libraries.

LGTM with a nit

Dec 13 2017, 11:54 PM · Restricted Project

Dec 5 2017

dvyukov accepted D40877: [TSan] Make more TSan interceptors symbolizer-aware..
Dec 5 2017, 10:42 PM

Dec 4 2017

dvyukov accepted D40726: Move __tsan::Vector to __sanitizer.
Dec 4 2017, 4:27 AM · Restricted Project

Dec 1 2017

dvyukov added inline comments to D40714: Correct atexit(3) support in MSan/NetBSD.
Dec 1 2017, 4:53 AM · Restricted Project

Nov 30 2017

dvyukov accepted D40665: [sanitizer] Implement NanoTime() on Darwin.
Nov 30 2017, 12:08 PM · Restricted Project
dvyukov accepted D40666: [sanitizer] Use MADV_FREE on Darwin/BSD to release pages to the OS.
Nov 30 2017, 12:08 PM · Restricted Project

Nov 29 2017

dvyukov added a comment to D40545: [sanitizer] add msvc atomic_compare_exchange_strong for sint32_t.

this is used in x-ray runtime , in xray_fdr_logging.cc ,such as line 56:

if (!__sanitizer::atomic_compare_exchange_strong(
         &LogFlushStatus, &Result, XRayLogFlushStatus::XRAY_LOG_FLUSHING,
         __sanitizer::memory_order_release))

currently x-ray didn't compile on windows,but this is a little step to make it build. I'm trying to build xray runtime on windows, some talks here:
http://lists.llvm.org/pipermail/llvm-dev/2017-November/119218.html

That's the one I referred to as "which should actually use atomic_uint32_t".
Please switch xray to uint32. I don't think we want to support all of 10 atomic operations X 6 memory orderings X 20 different types, that's more than 1000 combinations.
One day we should finally switch to C++ atomics, but until then it does not make sense to grow set of atomic operations infinitely.

I'm curious though, since that type is coercing an enum into an int, which is the only defined conversion as far as I can tell. While it "should" be safe to use atomic_uint32_t instead, I'm wondering whether it's actually safe to do that. If it is, then I'm happy with using atomic_uint32_t instead of an atomic_int32_t in XRay.

Nov 29 2017, 6:16 AM · Restricted Project
dvyukov added a comment to D40545: [sanitizer] add msvc atomic_compare_exchange_strong for sint32_t.

this is used in x-ray runtime , in xray_fdr_logging.cc ,such as line 56:

if (!__sanitizer::atomic_compare_exchange_strong(
         &LogFlushStatus, &Result, XRayLogFlushStatus::XRAY_LOG_FLUSHING,
         __sanitizer::memory_order_release))

currently x-ray didn't compile on windows,but this is a little step to make it build. I'm trying to build xray runtime on windows, some talks here:
http://lists.llvm.org/pipermail/llvm-dev/2017-November/119218.html

Nov 29 2017, 5:50 AM · Restricted Project

Nov 28 2017

dvyukov added inline comments to D40583: Defer StartBackgroundThread() and StopBackgroundThread() in TSan.
Nov 28 2017, 11:53 PM · Restricted Project
dvyukov added inline comments to D40583: Defer StartBackgroundThread() and StopBackgroundThread() in TSan.
Nov 28 2017, 11:52 PM · Restricted Project
dvyukov accepted D40583: Defer StartBackgroundThread() and StopBackgroundThread() in TSan.

LGTM with a nit

Nov 28 2017, 11:51 PM · Restricted Project
dvyukov accepted D40337: Support the setjmp(3) family of functions in TSan/NetBSD.
Nov 28 2017, 9:26 AM · Restricted Project
dvyukov added a comment to D40545: [sanitizer] add msvc atomic_compare_exchange_strong for sint32_t.

Where do you want to use this?
I see only 2 uses of atomic_sint32_t, at least one of which should actually use atomic_uint32_t.

Nov 28 2017, 4:52 AM · Restricted Project
dvyukov added a comment to D40337: Support the setjmp(3) family of functions in TSan/NetBSD.

I will give it a try.

Can I extract the LongJmp change and commit to the main sources now as it?

Nov 28 2017, 3:11 AM · Restricted Project
dvyukov added inline comments to D40337: Support the setjmp(3) family of functions in TSan/NetBSD.
Nov 28 2017, 2:44 AM · Restricted Project