dvyukov (Dmitry Vyukov)
User

Projects

User does not belong to any projects.

User Details

User Since
Dec 6 2012, 2:31 AM (285 w, 2 d)

Recent Activity

Yesterday

dvyukov added inline comments to D47289: [scudo] Improve the scalability of the shared TSD model.
Sat, May 26, 1:28 AM

Wed, May 16

dvyukov accepted D46961: [scudo] Quarantine optimization.

I see. Let's wait for C++14.

Wed, May 16, 10:59 AM
dvyukov added a comment to D46961: [scudo] Quarantine optimization.

Making it static const makes the compiler use cxa_guard_acquire & cxa_guard_release which we don't want to be able to stay free of libc++ dependencies.

Wed, May 16, 10:45 AM
dvyukov added a comment to D46961: [scudo] Quarantine optimization.
  • const uptr BatchClassId = SizeClassMap::ClassID(sizeof(QuarantineBatch));
Wed, May 16, 10:22 AM

Sun, May 13

dvyukov accepted D46793: [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock.
Sun, May 13, 1:38 AM

Fri, May 11

dvyukov added a comment to D46793: [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock.

Interesting. Who calls these functions? Why? Can we change them to call normal functions instead?

Fri, May 11, 11:57 PM

Wed, May 9

dvyukov accepted D46597: [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms.
Wed, May 9, 9:10 AM
dvyukov added inline comments to D46597: [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms.
Wed, May 9, 1:07 AM

Mon, May 7

dvyukov added a comment to D44881: [sanitizer_common] Ignore unloading of suppressed library.

There are several reasons:

  1. The address range is still ignored after unload. So if anything else loaded at that range it will be falsely ignore and this can lead to both false positives and false negatives.
  2. The list of ignored libraries is append-only, so it's not possible to remove from it.
  3. If a library is unloaded, it can be loaded again later. This will lead to unbounded growth of the list of ignored libraries. The list is fixed size, so soon we will crash with an obscure CHECK failure.
Mon, May 7, 11:38 AM

Mon, Apr 30

dvyukov committed rCRT331163: tsan: disable trace switching after multithreaded fork.
tsan: disable trace switching after multithreaded fork
Mon, Apr 30, 12:33 AM
dvyukov committed rL331163: tsan: disable trace switching after multithreaded fork.
tsan: disable trace switching after multithreaded fork
Mon, Apr 30, 12:32 AM

Apr 27 2018

dvyukov committed rCRT331023: tsan: improve "destroy of a locked mutex" reports.
tsan: improve "destroy of a locked mutex" reports
Apr 27 2018, 2:03 AM
dvyukov committed rL331023: tsan: improve "destroy of a locked mutex" reports.
tsan: improve "destroy of a locked mutex" reports
Apr 27 2018, 2:03 AM

Apr 23 2018

dvyukov added a comment to D45646: [tsan] Zero out the shadow memory for the stack and TLS in ThreadFinish.

I'm not too worried about the first variation. It's very unlikely that user code will try to mmap(MAP_FIXED)

Apr 23 2018, 2:47 AM · Restricted Project

Apr 20 2018

dvyukov added a comment to D45838: [sanitizer] More dead code removal.

Then I don't understand what happens here anymore. Go and drop it.

Apr 20 2018, 10:02 AM
dvyukov added a comment to D45838: [sanitizer] More dead code removal.

Tests are not linked with interceptors?

I didn't find anything that suggested that this particular one was, but someone else can probably chime in on that.
sanitizer::SanitizerSetThreadName is present in other binaries (Asan-x86-64-calls-Tests for example) and calls interceptor_prctl there, but is not called by anything.

Apr 20 2018, 9:27 AM
dvyukov added a comment to D45838: [sanitizer] More dead code removal.

Tests are not linked with interceptors?

Apr 20 2018, 9:12 AM
dvyukov added a comment to D45838: [sanitizer] More dead code removal.

I am not following. Isn't it a test for prctl interceptor? Why are we removing useful tests?
Perhaps it's not the most critical functionality. And perhaps SanitizerSetThreadName needs to be moved to the test itself. But still it's a test.

Apr 20 2018, 12:55 AM

Apr 19 2018

dvyukov added a comment to D45646: [tsan] Zero out the shadow memory for the stack and TLS in ThreadFinish.

Let's concentrate on correctness for now.
Shadow memory is not guaranteed to be zeros for new memory. At munmap does not clear it, and probably some other things. Perhaps if you use munmap in test, it will be easier to catch this (e.g. mmap a large chunk of memory, write to it, munmap, create a thread).

Apr 19 2018, 2:10 AM · Restricted Project
dvyukov committed rCRT330312: tsan: fix compiler warnings.
tsan: fix compiler warnings
Apr 19 2018, 12:46 AM
dvyukov committed rL330312: tsan: fix compiler warnings.
tsan: fix compiler warnings
Apr 19 2018, 12:46 AM

Apr 16 2018

dvyukov committed rCRT330122: tsan: add support for linux/powerpc64 in buildgo.sh.
tsan: add support for linux/powerpc64 in buildgo.sh
Apr 16 2018, 4:51 AM
dvyukov committed rL330122: tsan: add support for linux/powerpc64 in buildgo.sh.
tsan: add support for linux/powerpc64 in buildgo.sh
Apr 16 2018, 4:47 AM
dvyukov closed D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Submitted in:
http://llvm.org/viewvc/llvm-project?view=revision&revision=330122
Thanks!

Apr 16 2018, 4:47 AM
dvyukov added a comment to D45646: [tsan] Zero out the shadow memory for the stack and TLS in ThreadFinish.

Two things that I don't like here:

  1. This imposes cost of zeroing of up to 32MB (standard 8MB stack x 4x shadow) per thread creation/destruction for all OSes. Some programs create threads like insane.
  2. I don't think this fixes the actual root cause, only makes it even harder to localize. Note that cur_thread_finalize already clears the shadow slot, so if pthread reuses stack/tls wholesale, then the slot should be zero already. However, tsan does not generally keep shadow clear (e.g. munmap does not clear shadow too, and most likely a bunch of other things). So if the slot reuses memory from a previous mmap, it will crash the same way.

I wonder if moving the slot to _meta_ shadow is the right things to do. We actually clear meta shadow on unmap. I don't see where we clear stack, but we should, otherwise we can leak lots of sync objects on stack.

Apr 16 2018, 2:09 AM · Restricted Project

Apr 13 2018

dvyukov accepted D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Looks good to me.
Do you have commit access, or you want me to commit it?

Apr 13 2018, 8:56 AM
dvyukov added a comment to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Much nicer now!
I think this is ready for commit after fixing these few nits.

Apr 13 2018, 12:56 AM

Apr 12 2018

dvyukov added inline comments to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.
Apr 12 2018, 3:11 AM

Apr 11 2018

dvyukov added a reviewer for D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh: dvyukov.
Apr 11 2018, 1:52 AM
dvyukov added inline comments to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.
Apr 11 2018, 1:51 AM

Apr 5 2018

dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

No, it explicitly checks alignment to decide whether to make a libcall itself.

Apr 5 2018, 9:41 AM
dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

I've just checked the LangRef and realised this is false. LLVM does make such loads & stores UB, though I still think the Clang use is valid (if unfortunate).

Apr 5 2018, 8:01 AM
dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

Atomics accessed via C11 _Atomic and C++11 std::atomic will be suitably aligned, but there's a reasonable amount of legacy code that uses the GCC builtins on non-atomic types (of unknown alignment) and this is what Clang uses to implement those accesses when they come up.

Apr 5 2018, 7:31 AM
dvyukov added a comment to D45321: [atomics] Fix runtime calls for misaligned atomics.

How how can atomic objects end up being mis-aligned? Isn't it UB already?

Apr 5 2018, 6:55 AM

Apr 3 2018

dvyukov added a comment to D44981: asan: kernel: make no_sanitize("address") attribute work with -fsanitize=kernel-address.

You are right, it's not the best idea.
Is it too late to get rid of C/C++-level "kernel-address" attribute entirely? For the source code perspective, it's just ASan, but for the kernel. If that's an important distinction, one can always do #ifdef KERNEL or something.

Apr 3 2018, 1:11 PM

Mar 21 2018

dvyukov added a comment to D44714: Change TSAN external symbolization API to support returning multiple frames.

This broke darwin build. I've fixed it up in:
http://llvm.org/viewvc/llvm-project?view=revision&revision=328082
Both external symbolization tests still pass, so I hope it works for your use case.

Mar 21 2018, 2:30 AM
dvyukov committed rL328082: tsan: fix darwin build after 328079.
tsan: fix darwin build after 328079
Mar 21 2018, 2:29 AM
dvyukov committed rCRT328082: tsan: fix darwin build after 328079.
tsan: fix darwin build after 328079
Mar 21 2018, 2:29 AM
dvyukov added a comment to D44714: Change TSAN external symbolization API to support returning multiple frames.

Submitted in rev 328079.

Mar 21 2018, 1:47 AM
dvyukov committed rL328079: tsan: support inlined frames in external symbolization.
tsan: support inlined frames in external symbolization
Mar 21 2018, 1:47 AM
dvyukov committed rCRT328079: tsan: support inlined frames in external symbolization.
tsan: support inlined frames in external symbolization
Mar 21 2018, 1:47 AM
dvyukov accepted D44714: Change TSAN external symbolization API to support returning multiple frames.

Looks good. Comitting.

Mar 21 2018, 1:43 AM

Mar 16 2018

dvyukov committed rL327700: tsan: revert: Update buildgo.sh to pass -isysroot on Darwin..
tsan: revert: Update buildgo.sh to pass -isysroot on Darwin.
Mar 16 2018, 3:23 AM
dvyukov committed rCRT327700: tsan: revert: Update buildgo.sh to pass -isysroot on Darwin..
tsan: revert: Update buildgo.sh to pass -isysroot on Darwin.
Mar 16 2018, 3:23 AM

Mar 15 2018

dvyukov accepted D44071: [TSan] fix Go runtime test on amd64 with PIE.

LGTM
It actually fails on my laptop in the same way now. But works with this patch.

Mar 15 2018, 3:20 AM

Mar 7 2018

dvyukov accepted D44246: [sanitizer] Generalize atomic_uint8_t, atomic_uint16_t, ... into a template. NFC..
Mar 7 2018, 11:01 PM · Restricted Project

Feb 14 2018

dvyukov accepted D43325: [TSan] Fix static TLS boundaries calculations in __tls_get_addr interceptor..
Feb 14 2018, 10:40 PM

Feb 7 2018

dvyukov added a comment to D43025: [tsan] Add support for linux/powerpc64 in buildgo.sh.

Is there an upstream Go bug for race on powerpc? Please CC me on it.

Feb 7 2018, 10:16 AM

Jan 29 2018

dvyukov committed rL323657: tsan: deflake a test.
tsan: deflake a test
Jan 29 2018, 7:12 AM
dvyukov closed D42633: tsan: deflake a test.

Submitted in r323657.

Jan 29 2018, 7:12 AM
dvyukov committed rCRT323657: tsan: deflake a test.
tsan: deflake a test
Jan 29 2018, 7:11 AM
dvyukov created D42633: tsan: deflake a test.
Jan 29 2018, 12:45 AM

Jan 22 2018

dvyukov closed D42384: asan: allow inline instrumentation for the kernel.

Committed in r323140.

Jan 22 2018, 11:08 AM
dvyukov committed rL323140: asan: allow inline instrumentation for the kernel.
asan: allow inline instrumentation for the kernel
Jan 22 2018, 11:08 AM
dvyukov accepted D42384: asan: allow inline instrumentation for the kernel.
Jan 22 2018, 10:40 AM

Dec 13 2017

dvyukov accepted D41190: [tsan] Separate the constants in libignore and bump the maximum for instrumented libraries.

LGTM with a nit

Dec 13 2017, 11:54 PM · Restricted Project

Dec 5 2017

dvyukov accepted D40877: [TSan] Make more TSan interceptors symbolizer-aware..
Dec 5 2017, 10:42 PM

Dec 4 2017

dvyukov accepted D40726: Move __tsan::Vector to __sanitizer.
Dec 4 2017, 4:27 AM · Restricted Project

Dec 1 2017

dvyukov added inline comments to D40714: Correct atexit(3) support in MSan/NetBSD.
Dec 1 2017, 4:53 AM · Restricted Project

Nov 30 2017

dvyukov accepted D40665: [sanitizer] Implement NanoTime() on Darwin.
Nov 30 2017, 12:08 PM · Restricted Project
dvyukov accepted D40666: [sanitizer] Use MADV_FREE on Darwin/BSD to release pages to the OS.
Nov 30 2017, 12:08 PM · Restricted Project

Nov 29 2017

dvyukov added a comment to D40545: [sanitizer] add msvc atomic_compare_exchange_strong for sint32_t.

this is used in x-ray runtime , in xray_fdr_logging.cc ,such as line 56:

if (!__sanitizer::atomic_compare_exchange_strong(
         &LogFlushStatus, &Result, XRayLogFlushStatus::XRAY_LOG_FLUSHING,
         __sanitizer::memory_order_release))

currently x-ray didn't compile on windows,but this is a little step to make it build. I'm trying to build xray runtime on windows, some talks here:
http://lists.llvm.org/pipermail/llvm-dev/2017-November/119218.html

That's the one I referred to as "which should actually use atomic_uint32_t".
Please switch xray to uint32. I don't think we want to support all of 10 atomic operations X 6 memory orderings X 20 different types, that's more than 1000 combinations.
One day we should finally switch to C++ atomics, but until then it does not make sense to grow set of atomic operations infinitely.

I'm curious though, since that type is coercing an enum into an int, which is the only defined conversion as far as I can tell. While it "should" be safe to use atomic_uint32_t instead, I'm wondering whether it's actually safe to do that. If it is, then I'm happy with using atomic_uint32_t instead of an atomic_int32_t in XRay.

Nov 29 2017, 6:16 AM · Restricted Project
dvyukov added a comment to D40545: [sanitizer] add msvc atomic_compare_exchange_strong for sint32_t.

this is used in x-ray runtime , in xray_fdr_logging.cc ,such as line 56:

if (!__sanitizer::atomic_compare_exchange_strong(
         &LogFlushStatus, &Result, XRayLogFlushStatus::XRAY_LOG_FLUSHING,
         __sanitizer::memory_order_release))

currently x-ray didn't compile on windows,but this is a little step to make it build. I'm trying to build xray runtime on windows, some talks here:
http://lists.llvm.org/pipermail/llvm-dev/2017-November/119218.html

Nov 29 2017, 5:50 AM · Restricted Project

Nov 28 2017

dvyukov added inline comments to D40583: Defer StartBackgroundThread() and StopBackgroundThread() in TSan.
Nov 28 2017, 11:53 PM · Restricted Project
dvyukov added inline comments to D40583: Defer StartBackgroundThread() and StopBackgroundThread() in TSan.
Nov 28 2017, 11:52 PM · Restricted Project
dvyukov accepted D40583: Defer StartBackgroundThread() and StopBackgroundThread() in TSan.

LGTM with a nit

Nov 28 2017, 11:51 PM · Restricted Project
dvyukov accepted D40337: Support the setjmp(3) family of functions in TSan/NetBSD.
Nov 28 2017, 9:26 AM · Restricted Project
dvyukov added a comment to D40545: [sanitizer] add msvc atomic_compare_exchange_strong for sint32_t.

Where do you want to use this?
I see only 2 uses of atomic_sint32_t, at least one of which should actually use atomic_uint32_t.

Nov 28 2017, 4:52 AM · Restricted Project
dvyukov added a comment to D40337: Support the setjmp(3) family of functions in TSan/NetBSD.

I will give it a try.

Can I extract the LongJmp change and commit to the main sources now as it?

Nov 28 2017, 3:11 AM · Restricted Project
dvyukov added inline comments to D40337: Support the setjmp(3) family of functions in TSan/NetBSD.
Nov 28 2017, 2:44 AM · Restricted Project
dvyukov accepted D40341: Handle symbol renaming of sigaction for NetBSD.
Nov 28 2017, 2:31 AM · Restricted Project
dvyukov added a comment to D40341: Handle symbol renaming of sigaction for NetBSD.

This patch has been tested only on NetBSD and with asan, msan, lsan, tsan, ubsan, libfuzzer, safestack, profile.

I request for a tester on Linux as it might have impact for e.g. esan.

Nov 28 2017, 2:31 AM · Restricted Project

Nov 26 2017

dvyukov accepted D40294: Prevent Thread Exited/Joined events race.
Nov 26 2017, 10:36 AM · Restricted Project
dvyukov accepted D40457: Detect thread termination in LSan/NetBSD.
Nov 26 2017, 12:12 AM · Restricted Project
dvyukov added a comment to D40456: Enable additonal features in NetBSD.

Evgenii, do you know what's the policy for enabling these flags?

Nov 26 2017, 12:07 AM · Restricted Project

Nov 25 2017

dvyukov accepted D40382: Plug dlerror() leak for swift_demangle.
Nov 25 2017, 4:41 AM · Restricted Project

Nov 21 2017

dvyukov added inline comments to D40294: Prevent Thread Exited/Joined events race.
Nov 21 2017, 6:54 AM · Restricted Project
dvyukov accepted D40262: Correct NetBSD support in pthread_once(3)/TSan.
Nov 21 2017, 1:30 AM · Restricted Project

Nov 20 2017

dvyukov added inline comments to D40262: Correct NetBSD support in pthread_once(3)/TSan.
Nov 20 2017, 12:24 PM · Restricted Project
dvyukov accepted D40251: Add DemangleFunctionName for backtracing on NetBSD.
Nov 20 2017, 8:19 AM · Restricted Project
dvyukov accepted D40243: Handle NetBSD specific indirection of libpthread functions.
Nov 20 2017, 8:12 AM · Restricted Project
dvyukov added a comment to D40243: Handle NetBSD specific indirection of libpthread functions.

There is an open question what to do with tests. They report results with the mangled name in stack trace, instead of the regular libpthread(3) ones.

$ ./mutex                                                                    
==================
WARNING: ThreadSanitizer: data race (pid=13274)
  Write of size 4 at 0x0000014f9ec0 by thread T1 (mutexes: write M76):
    #0 Thread1(void*) /public/llvm-build/mutex.cc:36:9 (mutex+0x46e5ec)

  Previous write of size 4 at 0x0000014f9ec0 by thread T2:
    #0 Thread2(void*) /public/llvm-build/mutex.cc:42:9 (mutex+0x46e6b7)

  Location is global 'Global' of size 4 at 0x0000014f9ec0 (mutex+0x0000014f9ec0)

  Mutex M76 (0x0000014f9ec8) created at:
    #0 __libc_mutex_init /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1137:3 (mutex+0x413ef3)
    #1 main /public/llvm-build/mutex.cc:66:3 (mutex+0x46e796)

  Thread T1 (tid=2, running) created by main thread at:
    #0 pthread_create /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:921:3 (mutex+0x412cf0)
    #1 main /public/llvm-build/mutex.cc:71:3 (mutex+0x46e7b5)

  Thread T2 (tid=3, finished) created by main thread at:
    #0 pthread_create /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:921:3 (mutex+0x412cf0)
    #1 main /public/llvm-build/mutex.cc:72:3 (mutex+0x46e7d4)

SUMMARY: ThreadSanitizer: data race /public/llvm-build/mutex.cc:36:9 in Thread1(void*)
==================
ThreadSanitizer: reported 1 warnings

__libc_mutex_init printed instead of pthread_mutex_init. This breaks tests that check for exact strings.

I will implement fix for tests (demangle names or change tests) in a dedicated review.

Nov 20 2017, 5:36 AM · Restricted Project
dvyukov accepted D40241: Handle NetBSD specific indirection of libpthread functions.
Nov 20 2017, 3:38 AM · Restricted Project

Nov 17 2017

dvyukov added a comment to D40159: Correct handling of the TLS/NetBSD block of the main program.

Joerg, please review.

Nov 17 2017, 12:02 AM · Restricted Project

Nov 15 2017

dvyukov accepted D40105: Implement GetTls() for NetBSD.
Nov 15 2017, 11:41 PM · Restricted Project
dvyukov added inline comments to D39935: [tsan] Fix signal chaining.
Nov 15 2017, 11:40 PM
dvyukov added a comment to D40032: [compiler-rt] Replace forkpty with posix_spawn.

I can't say that I fully understand what happens here, but I can rubber stamp LGTM if you wish.

Nov 15 2017, 12:01 AM · Restricted Project

Nov 14 2017

dvyukov added inline comments to D39935: [tsan] Fix signal chaining.
Nov 14 2017, 12:06 AM

Nov 13 2017

dvyukov accepted D39924: [PowerPC][tsan] Update tsan to handle changed memory layouts in newer kernels.
Nov 13 2017, 12:05 AM
dvyukov accepted D39935: [tsan] Fix signal chaining.

LGTM with a nit

Nov 13 2017, 12:03 AM

Nov 12 2017

dvyukov accepted D39929: [tsan] Deadly signal handler for tsan.
Nov 12 2017, 11:55 PM

Nov 8 2017

dvyukov accepted D39619: Correct atexit(3) support in TSan/NetBSD.
Nov 8 2017, 8:22 AM · Restricted Project
dvyukov added inline comments to D39619: Correct atexit(3) support in TSan/NetBSD.
Nov 8 2017, 8:01 AM · Restricted Project
dvyukov added a comment to D39619: Correct atexit(3) support in TSan/NetBSD.
Nov 8 2017, 7:59 AM · Restricted Project

Nov 7 2017

dvyukov added inline comments to D39619: Correct atexit(3) support in TSan/NetBSD.
Nov 7 2017, 11:55 PM · Restricted Project
dvyukov committed rL317587: tsan: allow usage of global vars with ctors in interceptors.
tsan: allow usage of global vars with ctors in interceptors
Nov 7 2017, 8:32 AM
dvyukov closed D39721: tsan: allow usage of global vars with ctors in interceptors.

Committed as 317587.

Nov 7 2017, 8:31 AM
dvyukov added inline comments to D39619: Correct atexit(3) support in TSan/NetBSD.
Nov 7 2017, 4:23 AM · Restricted Project
dvyukov added a comment to D39619: Correct atexit(3) support in TSan/NetBSD.

Hmm, I might be doing something incorrectly, but there is a problem with Vector. We fire the destructor for this LIFO container.. before actually calling atexit(3) functions from libc.

Nov 7 2017, 2:26 AM · Restricted Project
dvyukov created D39721: tsan: allow usage of global vars with ctors in interceptors.
Nov 7 2017, 2:22 AM