Page MenuHomePhabricator

[sanitizers] [windows] Use InternalMmapVector instead of silencing -Wframe-larger-than
ClosedPublic

Authored by mstorsjo on Mar 1 2021, 1:27 PM.

Details

Summary

Also use this in ReadBinaryName which currently is producing warnings.

Keep pragmas for silencing warnings in sanitizer_unwind_win.cpp, as that can be called more frequently.

I haven't run tests for this (I don't have a setup yet where they'd run), but I'd appreciate if someone with such a setup available could run them.

Diff Detail

Event Timeline

mstorsjo requested review of this revision.Mar 1 2021, 1:27 PM
mstorsjo created this revision.
Herald added a project: Restricted Project. · View Herald TranscriptMar 1 2021, 1:27 PM
Herald added a subscriber: Restricted Project. · View Herald Transcript
mstorsjo updated this revision to Diff 327275.Mar 1 2021, 1:46 PM

Updated patch that actually does apply on main instead of on top of the other one (from D97579).

vitalybuka added inline comments.
compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_win.cpp
140–141

Can you do InternalMmapVector<PSYMBOL_INFO> ?

mstorsjo added inline comments.Mar 1 2021, 2:04 PM
compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_win.cpp
140–141

Not really, this a struct followed by a flexible array.

vitalybuka added inline comments.Mar 1 2021, 2:54 PM
compiler-rt/lib/sanitizer_common/sanitizer_unwind_win.cpp
50

This called unwind slow and usually called infrequently, but user can
set fast_unwind_on_malloc=false and make it frequent.
So it's not nice that we make it slower.

You can try:

NOINLINE
static void UnwindSlowInternal(uptr pc, void *context, u32 max_depth, STACKFRAME64*) {
}

void BufferedStackTrace::UnwindSlow(uptr pc, void *context, u32 max_depth) {
  CONTEXT ctx = *(CONTEXT *)context;
  STACKFRAME64 stack_frame;
  UnwindSlowInternal(...., &stack_frame)
}

And also I don't know if all these are worth of hassle. Maybe we can just raise limit for -Wframe-larger-than=

rnk added inline comments.Mar 1 2021, 3:31 PM
compiler-rt/lib/sanitizer_common/sanitizer_unwind_win.cpp
50

Windows doesn't use the fast unwind path at all, so actually we come here every time we capture a stack trace for a heap event. See:
https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/sanitizer_common/sanitizer_stacktrace.h#L26

#elif SANITIZER_WINDOWS
# define SANITIZER_CAN_FAST_UNWIND 0

There are probably much faster ways to walk the stack, if someone was really motivated to maintain Windows ASan.

For now we should probably just revert this change. All the other changes seem like they should happen at most once: either during startup or during error reporting, so I'd keep them.

mstorsjo updated this revision to Diff 327365.Mar 1 2021, 11:42 PM
mstorsjo edited the summary of this revision. (Show Details)

Removed the changes to sanitizer_unwind_win.cpp, applied the suggested clang-format.

mcgov added a subscriber: cbezault.Mar 3 2021, 12:31 PM

Can we add @cbezault to this? He's the DevDiv vclibs person who owns Windows ASan going forward

Can someone with a working windows sanitizers try this one out?

Can someone with a working windows sanitizers try this one out?

I applied this to cf806d91d562736192f91478eb47daa29c2230bf, built with VS2019 and ran some tests.

ninja check-sanitizer was okay
ninja check-ubsan had a few failures

These weren't present with the patch.

FAIL: UBSan-Standalone-x86_64 :: TestCases/Integer/suppressions.cpp (31 of 82)
******************** TEST 'UBSan-Standalone-x86_64 :: TestCases/Integer/suppressions.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';      F:/git/llvm-project/stage1/./bin/clang.exe      -fsanitize=integer -g0 F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp -o F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp
: 'RUN: at line 11';   env UBSAN_OPTIONS=halt_on_error=1 not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp
: 'RUN: at line 13';   echo "signed-integer-overflow:F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp" > F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.wrong-supp
: 'RUN: at line 14';   env UBSAN_OPTIONS=halt_on_error=1:suppressions='"F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.wrong-supp"' not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp
: 'RUN: at line 16';   echo "unsigned-integer-overflow:do_overflow" > F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.func-supp
: 'RUN: at line 17';   env UBSAN_OPTIONS=halt_on_error=1:suppressions='"F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.func-supp"'  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp
: 'RUN: at line 21';   echo "unsigned-integer-overflow:suppressions.cpp.tmp" > F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.module-supp
: 'RUN: at line 22';   env UBSAN_OPTIONS=halt_on_error=1:suppressions='"F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.module-supp"'  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp
: 'RUN: at line 25';   echo "unsigned-integer-overflow:F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp" > F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.file-supp
: 'RUN: at line 26';   env UBSAN_OPTIONS=halt_on_error=1:suppressions='"F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.file-supp"'  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp
: 'RUN: at line 29';      F:/git/llvm-project/stage1/./bin/clang.exe      -fsanitize=integer -fno-sanitize-recover=integer F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp -o F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp-norecover
: 'RUN: at line 30';   env UBSAN_OPTIONS=halt_on_error=1:suppressions='"F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.module-supp"' not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp-norecover 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "F:/git/llvm-project/stage1/./bin/clang.exe" "-fsanitize=integer" "-g0" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp" "-o" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp"
# command output:
   Creating library F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.lib and object F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.exp

$ ":" "RUN: at line 11"
$ "env" "UBSAN_OPTIONS=halt_on_error=1" "not" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp"
$ ":" "RUN: at line 13"
$ "echo" "signed-integer-overflow:F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp"
$ ":" "RUN: at line 14"
$ "env" "UBSAN_OPTIONS=halt_on_error=1:suppressions="F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.wrong-supp"" "not" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp"
$ ":" "RUN: at line 16"
$ "echo" "unsigned-integer-overflow:do_overflow"
$ ":" "RUN: at line 17"
$ "env" "UBSAN_OPTIONS=halt_on_error=1:suppressions="F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp.func-supp"" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Integer\Output\suppressions.cpp.tmp"
# command stderr:
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp:35:44: runtime error: unsigned integer overflow: 10000000000000000000 + 9000000000000000000 cannot be represented in type 'unsigned long long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Integer\suppressions.cpp:35:44 in

error: command failed with exit status: 1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50..
FAIL: UBSan-Standalone-x86_64 :: TestCases/Misc/missing_return.cpp (50 of 82)
******************** TEST 'UBSan-Standalone-x86_64 :: TestCases/Misc/missing_return.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';      F:/git/llvm-project/stage1/./bin/clang.exe      -fsanitize=return -gline-tables-only -gcodeview -gcolumn-info F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp -O3 -o F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp
: 'RUN: at line 2';   not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp
: 'RUN: at line 3';   env UBSAN_OPTIONS=print_stacktrace=1 not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp --check-prefix=CHECK-STACKTRACE
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "F:/git/llvm-project/stage1/./bin/clang.exe" "-fsanitize=return" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp" "-O3" "-o" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp"
# command output:
   Creating library F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.lib and object F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.exp

# command stderr:
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:8:24: warning: GCC does not allow 'noinline' attribute in this position on a function definition [-Wgcc-compat]
int f() __attribute__((noinline)) {
                       ^
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:10:1: warning: non-void function does not return a value [-Wreturn-type]
}
^
2 warnings generated.

$ ":" "RUN: at line 2"
$ "not" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp"
$ ":" "RUN: at line 3"
$ "env" "UBSAN_OPTIONS=print_stacktrace=1" "not" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp" "--check-prefix=CHECK-STACKTRACE"
# command stderr:
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:9:22: error: CHECK-STACKTRACE: expected string not found in input
// CHECK-STACKTRACE: #0 {{.*}}f{{.*}}missing_return.cpp:[[@LINE-1]]
                     ^
<stdin>:1:1: note: scanning from here
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:8:5: runtime error: execution reached the end of a value-returning function without returning a value
^
<stdin>:1:1: note: with "@LINE-1" equal to "8"
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:8:5: runtime error: execution reached the end of a value-returning function without returning a value
^
<stdin>:1:46: note: possible intended match here
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:8:5: runtime error: execution reached the end of a value-returning function without returning a value
                                             ^

Input file: <stdin>
Check file: F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\Misc\missing_return.cpp:8:5: runtime error: execution reached the end of a value-returning function without returning a value
check:9'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:9'1                                                                                                                                                                                         with "@LINE-1" equal to "8"
check:9'2                                                  ?                                                                                                                                      possible intended match
           2:  #0 0x7ff7d55c80cf (F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp+0x80cf)
check:9'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           3:  #1 0x7ff7d55c80e8 (F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp+0x80e8)
check:9'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           4:  #2 0x7ff7d55cb02b (F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\Misc\Output\missing_return.cpp.tmp+0xb02b)
check:9'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           5:  #3 0x7ffc0a8b7033 (C:\WINDOWS\System32\KERNEL32.DLL+0x17033)
check:9'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           6:  #4 0x7ffc0c69d240 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x4d240)
check:9'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           .
           .
           .
>>>>>>

error: command failed with exit status: 1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.
FAIL: UBSan-Standalone-x86_64 :: TestCases/TypeCheck/misaligned.cpp (81 of 82)
******************** TEST 'UBSan-Standalone-x86_64 :: TestCases/TypeCheck/misaligned.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';      F:/git/llvm-project/stage1/./bin/clang.exe      -gline-tables-only -gcodeview -gcolumn-info -fsanitize=alignment F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp -O3 -o F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp
: 'RUN: at line 2';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp l0 &&  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp s0 &&  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp r0 &&  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp m0 &&  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp f0 &&  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp n0 &&  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp u0
: 'RUN: at line 3';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp l1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-LOAD --strict-whitespace
: 'RUN: at line 4';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp r1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-REFERENCE
: 'RUN: at line 5';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp m1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-MEMBER
: 'RUN: at line 6';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp f1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-MEMFUN
: 'RUN: at line 7';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp n1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-NEW
: 'RUN: at line 8';    F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp u1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-UPCAST
: 'RUN: at line 9';   env UBSAN_OPTIONS=print_stacktrace=1  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp l1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-LOAD --check-prefix=CHECK-STACK-LOAD
: 'RUN: at line 11';      F:/git/llvm-project/stage1/./bin/clang.exe      -fsanitize=alignment -fno-sanitize-recover=alignment F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp -O3 -o F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp
: 'RUN: at line 12';   not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp s1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-STORE
: 'RUN: at line 13';   not  F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp w1 2>&1 | FileCheck F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp --check-prefix=CHECK-WILD
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "F:/git/llvm-project/stage1/./bin/clang.exe" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-fsanitize=alignment" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "-O3" "-o" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp"
# command output:
   Creating library F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.lib and object F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.exp

# command stderr:
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:47:15: warning: use of logical '&&' with constant operand [-Wconstant-logical-operand]
    return *p && 0;
              ^  ~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:47:15: note: use '&' for a bitwise operation
    return *p && 0;
              ^~
              &
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:47:15: note: remove constant to silence this warning
    return *p && 0;
             ~^~~~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:71:17: warning: use of logical '&&' with constant operand [-Wconstant-logical-operand]
    return s->k && 0;
                ^  ~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:71:17: note: use '&' for a bitwise operation
    return s->k && 0;
                ^~
                &
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:71:17: note: remove constant to silence this warning
    return s->k && 0;
               ~^~~~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:78:19: warning: use of logical '&&' with constant operand [-Wconstant-logical-operand]
    return s->f() && 0;
                  ^  ~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:78:19: note: use '&' for a bitwise operation
    return s->f() && 0;
                  ^~
                  &
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:78:19: note: remove constant to silence this warning
    return s->f() && 0;
                 ~^~~~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:85:27: warning: use of logical '&&' with constant operand [-Wconstant-logical-operand]
    return (new (s) S)->k && 0;
                          ^  ~
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:85:27: note: use '&' for a bitwise operation
    return (new (s) S)->k && 0;
                          ^~
                          &
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:85:27: note: remove constant to silence this warning
    return (new (s) S)->k && 0;
                         ~^~~~
4 warnings generated.

$ ":" "RUN: at line 2"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "l0"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "s0"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "r0"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "m0"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "f0"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "n0"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "u0"
$ ":" "RUN: at line 3"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "l1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-LOAD" "--strict-whitespace"
$ ":" "RUN: at line 4"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "r1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-REFERENCE"
$ ":" "RUN: at line 5"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "m1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-MEMBER"
$ ":" "RUN: at line 6"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "f1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-MEMFUN"
$ ":" "RUN: at line 7"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "n1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-NEW"
$ ":" "RUN: at line 8"
$ "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "u1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-UPCAST"
$ ":" "RUN: at line 9"
$ "env" "UBSAN_OPTIONS=print_stacktrace=1" "F:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\Standalone-x86_64\TestCases\TypeCheck\Output\misaligned.cpp.tmp" "l1"
$ "FileCheck" "F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp" "--check-prefix=CHECK-LOAD" "--check-prefix=CHECK-STACK-LOAD"
# command stderr:
F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:48:23: error: CHECK-STACK-LOAD: expected string not found in input
 // CHECK-STACK-LOAD: #0 {{.*}}main{{.*}}misaligned.cpp
                      ^
<stdin>:4:3: note: scanning from here
 ^
  ^
<stdin>:5:109: note: possible intended match here
 #0 0x7ff6a5118273 (f:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\standalone-x86_64\testcases\typecheck\output\misaligned.cpp.tmp+0x8273)
                                                                                                            ^

Input file: <stdin>
Check file: F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            1: F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:47:12: runtime error: load of misaligned address 0x00bbbe32f9c5 for type 'int', which requires 4 byte alignment
            2: 0x00bbbe32f9c5: note: pointer points here
            3:  00 00 00 01 02 03 04 05 00 00 00 00 00 00 00 20 aa 71 d2 79 02 00 00 f0 e4 72 d2 79 02 00 00 00
            4:  ^
check:48'0       X~ error: no match found
            5:  #0 0x7ff6a5118273 (f:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\standalone-x86_64\testcases\typecheck\output\misaligned.cpp.tmp+0x8273)
check:48'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:48'1                                                                                                                 ?                                           possible intended match
            6:  #1 0x7ff6a511b27b (f:\git\llvm-project\stage1\projects\compiler-rt\test\ubsan\standalone-x86_64\testcases\typecheck\output\misaligned.cpp.tmp+0xb27b)
check:48'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            7:  #2 0x7ffc0a8b7033 (C:\WINDOWS\System32\KERNEL32.DLL+0x17033)
check:48'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            8:  #3 0x7ffc0c69d240 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x4d240)
check:48'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            9:
check:48'0     ~
           10: SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:\git\llvm-project\compiler-rt\test\ubsan\TestCases\TypeCheck\misaligned.cpp:47:12 in
check:48'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>

error: command failed with exit status: 1

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
********************
Failed Tests (3):
  UBSan-Standalone-x86_64 :: TestCases/Integer/suppressions.cpp
  UBSan-Standalone-x86_64 :: TestCases/Misc/missing_return.cpp
  UBSan-Standalone-x86_64 :: TestCases/TypeCheck/misaligned.cpp


Testing Time: 46.90s
  Unsupported      : 22
  Passed           : 55
  Expectedly Failed:  2
  Failed           :  3

Can someone with a working windows sanitizers try this one out?

I applied this to cf806d91d562736192f91478eb47daa29c2230bf, built with VS2019 and ran some tests.

Thanks, much appreciated!

These weren't present with the patch.

I presume you meant weren't present without the patch?

Failed Tests (3):
  UBSan-Standalone-x86_64 :: TestCases/Integer/suppressions.cpp
  UBSan-Standalone-x86_64 :: TestCases/Misc/missing_return.cpp
  UBSan-Standalone-x86_64 :: TestCases/TypeCheck/misaligned.cpp

Can you pinpoint which one of the couple changes that is the one that breaks it? There's roughly four cases of buffers converted from plain stack buffer to InternalMmapVector with a bit of syntax changes around each.

These weren't present with the patch.

I presume you meant weren't present without the patch?

That's right, my typo. These weren't present without the patch.

Can you pinpoint which one of the couple changes that is the one that breaks it? There's roughly four cases of buffers converted from plain stack buffer to InternalMmapVector with a bit of syntax changes around each.

It appears to be the change in GetPreferredBase. Reverting just that change causes those ubsan tests to pass.

There are also a lot of failures with check-asan, I think undoing the change in GetPreferredBase fixes those as well.

Thanks! That narrowed it down nicely.

compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
604

Ah, this sizeof(buf) needs to be buf.size() or something like that.

rnk added a comment.Mar 9 2021, 12:00 PM

(Thanks for running the tests, I started patching this in and setting up the build but didn't finish)

mstorsjo updated this revision to Diff 329434.Mar 9 2021, 12:20 PM

Fixed one missed case of sizeof(buf)

With an additional change from sizeof(buf) to buf.size() this fixes the check-asan and check-ubsan tests.

Note that there are 5 check-asan tests which fail for me with and without this patch:

AddressSanitizer-x86_64-windows :: TestCases/Windows/delay_dbghelp.cpp
AddressSanitizer-x86_64-windows :: TestCases/Windows/dll_host.cpp
AddressSanitizer-x86_64-windows :: TestCases/Windows/interface_symbols_windows.cpp
AddressSanitizer-x86_64-windows :: TestCases/Windows/iostream_sbo.cpp
AddressSanitizer-x86_64-windows :: TestCases/Windows/symbols_path.cpp

I don't have time to dig into these at the moment.

compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
605

I think that this line needs changing from sizeof(buf) to buf.size() as well.

mstorsjo updated this revision to Diff 329568.Mar 10 2021, 1:22 AM

Fixed another missed case of sizeof(buf) -> buf.size()

mstorsjo marked an inline comment as done.Mar 10 2021, 1:24 AM
mstorsjo added inline comments.
compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
605

Doh, thanks for catching it!, fixed now.

mstorsjo marked an inline comment as done.Mar 15 2021, 8:47 AM

@rnk - Does this look ok to you now? AFAIK it passes as many tests as before according to @russell.gallop.

rnk added inline comments.Mar 15 2021, 10:59 AM
compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
649

This might be a micro-optimization, but InternalMmapVector is built on VirtualAlloc, which is expensive, and now it's being done three times in a loop over the DLLs in a process. That might matter for startup time. Can you hoist the vectors out of the loop? You'll have to pass in memory to GetPreferredBase, which leaks implementation details, but I think it's worth doing.

mstorsjo updated this revision to Diff 330926.Mar 16 2021, 3:51 AM

Hoisted three InternalMmapVector out of a loop. I've managed to set up an environment where I can run the tests now, and the tests still pass as they did before.

rnk accepted this revision.Mar 16 2021, 11:40 AM

lgtm

This revision is now accepted and ready to land.Mar 16 2021, 11:40 AM