This is an archive of the discontinued LLVM Phabricator instance.

[compiler-rt][interception][win] Don't crash on unknown instructions
ClosedPublic

Authored by alvinhochun on Apr 30 2023, 6:55 AM.

Details

Summary

Do not treat unknown instructions as a fatal error. In most cases,
failure to intercept a function is reported by the caller, though
requires setting verbosity to 1 or higher to be visible.

Better error message reporting for asan will be added in a separate
patch.

Diff Detail

Event Timeline

alvinhochun created this revision.Apr 30 2023, 6:55 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 30 2023, 6:55 AM
Herald added subscribers: Enna1, dberris. · View Herald Transcript
alvinhochun requested review of this revision.Apr 30 2023, 6:55 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 30 2023, 6:55 AM
Herald added a subscriber: Restricted Project. · View Herald Transcript
vitalybuka accepted this revision.Apr 30 2023, 10:36 PM
This revision is now accepted and ready to land.Apr 30 2023, 10:36 PM
This revision was landed with ongoing or failed builds.May 4 2023, 7:41 AM
This revision was automatically updated to reflect the committed changes.
ayzhao added a subscriber: ayzhao.May 10 2023, 3:18 PM

We're starting to see build failures due to interception_failure_test.cpp unexpectedly passing on Windows: https://crbug.com/1443891

-- Testing: 71841 tests, 32 workers --
 
Testing: 
 XPASS: AddressSanitizer-x86_64-windows :: TestCases/interception_failure_test.cpp (398 of 71841)
 ******************** TEST 'AddressSanitizer-x86_64-windows :: TestCases/interception_failure_test.cpp' FAILED ********************
 Script:
 --
 : 'RUN: at line 4';      C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe  -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -gcodeview -gcolumn-info   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -O0 C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp -o C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp &&  C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp 2>&1 | FileCheck C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp
 : 'RUN: at line 5';      C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe  -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -gcodeview -gcolumn-info   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -O1 C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp -o C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp &&  C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp 2>&1 | FileCheck C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp
 : 'RUN: at line 6';      C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe  -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -gcodeview -gcolumn-info   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -O2 C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp -o C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp &&  C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp 2>&1 | FileCheck C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp
 : 'RUN: at line 7';      C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe  -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -gcodeview -gcolumn-info   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -O3 C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp -o C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp &&  C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp 2>&1 | FileCheck C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp
 --
 Exit Code: 0
 
 Command Output (stdout):
 --
 $ ":" "RUN: at line 4"
 $ "C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe" "-fsanitize=address" "-mno-omit-leaf-frame-pointer" "-fno-omit-frame-pointer" "-fno-optimize-sibling-calls" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-Wthread-safety" "-Wthread-safety-reference" "-Wthread-safety-beta" "-O0" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp" "-o" "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 # command output:
    Creating library C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.lib and object C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.exp
 
 $ "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 $ "FileCheck" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp"
 $ ":" "RUN: at line 5"
 $ "C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe" "-fsanitize=address" "-mno-omit-leaf-frame-pointer" "-fno-omit-frame-pointer" "-fno-optimize-sibling-calls" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-Wthread-safety" "-Wthread-safety-reference" "-Wthread-safety-beta" "-O1" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp" "-o" "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 # command output:
    Creating library C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.lib and object C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.exp
 
 $ "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 $ "FileCheck" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp"
 $ ":" "RUN: at line 6"
 $ "C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe" "-fsanitize=address" "-mno-omit-leaf-frame-pointer" "-fno-omit-frame-pointer" "-fno-optimize-sibling-calls" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-Wthread-safety" "-Wthread-safety-reference" "-Wthread-safety-beta" "-O2" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp" "-o" "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 # command output:
    Creating library C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.lib and object C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.exp
 
 $ "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 $ "FileCheck" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp"
 $ ":" "RUN: at line 7"
 $ "C:/b/s/w/ir/cache/builder/src/third_party/llvm-build/Release+Asserts/./bin/clang.exe" "-fsanitize=address" "-mno-omit-leaf-frame-pointer" "-fno-omit-frame-pointer" "-fno-optimize-sibling-calls" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-Wthread-safety" "-Wthread-safety-reference" "-Wthread-safety-beta" "-O3" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp" "-o" "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 # command output:
    Creating library C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.lib and object C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.exp
 
 $ "C:\b\s\w\ir\cache\builder\src\third_party\llvm-build\Release+Asserts\runtimes\runtimes-x86_64-pc-windows-msvc-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
 $ "FileCheck" "C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\test\asan\TestCases\interception_failure_test.cpp"
 
 --
 
 ********************
 
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
 ********************
 Unexpectedly Passed Tests (1):
   AddressSanitizer-x86_64-windows :: TestCases/interception_failure_test.cpp
 
 
 Testing Time: 506.20s
   Skipped            :    63
   Unsupported        : 11868
   Passed             : 81387
   Expectedly Failed  :   208
   Unexpectedly Passed:     1

We're starting to see build failures due to interception_failure_test.cpp unexpectedly passing on Windows: https://crbug.com/1443891

Hmmm - I guess it's concievable that this makes it no longer crash where it used to, so in that case the right course of action would be to remove the XFAIL in this case? The message for the XFAIL seems to be something different than what we're dealing with here though - but it's possible that the testcase, before this patch, used to fail for a new/different reason than what it was originally marked as XFAIL for.

Weirdly, that test XFAILs here on my i686-windows-msvc environment with check-asan (static runtime) because of different reasons:

  • Before the parent patch D148990, it would fail because interception_win tries to hook the strtol function defined in the test source (despite the comment in the test saying that asan doesn't intercept it), but the function contains the unhandled instruction 89 e5.
  • After D148990, it now fails because the function is intercepted by asan, and asan calls strtol with endptr being non-null, which causes the strlen inside the if block to be called and it correctly detects a heap use-after-free.

It is possible that, on x86_64 static runtime asan was simply failing to hook the strtol function because of unhandled instructions, and this patch making it non-fatal means the test now happens to XPASS but for the completely wrong reason.

The stated XFAIL reason "defining strtoll in a static build results in linker errors" certainly does not seem to apply nowadays. I don't know the rationale behind having this test, but it seems to me should just be marked UNSUPPORTED for win32-static-asan, because sanitizers on Windows certainly do try to intercept statically-linked copies of libc functions, be it user-provided or not [1].

[1]: https://github.com/llvm/llvm-project/blob/eea5d9cc4188584cbfdc18a8bac5316596e70263/compiler-rt/lib/interception/interception_win.h#L74-L77

I tried looking into this, but couldn't seem to reproduce it; I did try in an MSVC build with -DLLVM_ENABLE_RUNTIMES=compiler-rt.

In order to look at the issue closer; in a setup where this does trigger failures, can you get the output from llvm-lit -a for this individual test from before when it was XFAILing?

If building with LLVM_ENABLE_PROJECTS=compiler-rt (where I understand you don't run into this issue), I run python bin/llvm-lit.py -a projects\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\interception_failure_test.cpp directly in the build directory, and get this output:

$ "C:/code/llvm-project/llvm/build-msvc/./bin/clang.exe" "-fsanitize=address" "-mno-omit-leaf-frame-pointer" "-fno-omit-frame-pointer" "-fno-optimize-sibling-calls" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-O0" "C:\code\llvm-project\compiler-rt\test\asan\TestCases\interception_failure_test.cpp" "-o" "C:\code\llvm-project\llvm\build-msvc\projects\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
# command output:
libucrt.lib(strtox.obj) : error LNK2005: strtol already defined in interception_failure_test-657bd4.o
   Creating library C:\code\llvm-project\llvm\build-msvc\projects\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.lib and object C:\code\llvm-project\llvm\build-msvc\projects\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.exp
C:\code\llvm-project\llvm\build-msvc\projects\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp : fatal error LNK1169: one or more multiply defined symbols found

This seems to be exactly the same issue as is explained in the XFAIL of the test.

In the case of the build with LLVM_ENABLE_RUNTIMES=compiler-rt, I run a similar command in <builddir>/runtimes/runtimes-bins, where I do python ../../bin/llvm-lit.py -a compiler-rt\test\asan\X86_64WindowsConfig\TestCases\interception_failure_test.cpp. There I currently get a quite similar error:

$ "C:/code/llvm-project/llvm/build-msvc/./bin/clang.exe" "-fsanitize=address" "-mno-omit-leaf-frame-pointer" "-fno-omit-frame-pointer" "-fno-optimize-sibling-calls" "-gline-tables-only" "-gcodeview" "-gcolumn-info" "-Wthread-safety" "-Wthread-safety-reference" "-Wthread-safety-beta" "-O0" "C:\code\llvm-project\compiler-rt\test\asan\TestCases\interception_failure_test.cpp" "-o" "C:\code\llvm-project\llvm\build-msvc\runtimes\runtimes-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp"
# command output:
libucrt.lib(strtox.obj) : error LNK2005: strtol already defined in interception_failure_test-665768.o
   Creating library C:\code\llvm-project\llvm\build-msvc\runtimes\runtimes-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.lib and object C:\code\llvm-project\llvm\build-msvc\runtimes\runtimes-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.exp
C:\code\llvm-project\llvm\build-msvc\runtimes\runtimes-bins\compiler-rt\test\asan\X86_64WindowsConfig\TestCases\Output\interception_failure_test.cpp.tmp : fatal error LNK1169: one or more multiply defined symbols found

But if that bisected commit started making this no longer fail, it looks like the test isn't failing for this particular reason for you at the moment.

@alvinhochun managed to pinpoint and guess the differing issue.

With WinSDK/UCRT 10.0.19041.0, linking fails, since references to __stdio_common_vfprintf pulls in output.obj from libucrt.lib, and this object file pulls in wcstol from strtox.obj, which then later triggers the duplicate definition.

With libucrt.lib from WinSDK 10.0.22621.0, this no longer happens. But the test still failed, fulfilling the XFAIL, with the following error:

AddressSanitizer: CHECK failed: asan_rtl.cpp:387 "((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, 0x0) (tid=16328)
    <empty stack>

After this patch, this no longer triggers errors, and the test runs successfully. So the original XFAIL condition is dependent on WinSDK version, but there was a second error which made the XFAIL still seem to behave correctly, until after this change.

I can't comment on the rest of whether it's expected that this error condition that previously aborted execution, now no longer triggers a failure.

@alvinhochun managed to pinpoint and guess the differing issue.

With WinSDK/UCRT 10.0.19041.0, linking fails, since references to __stdio_common_vfprintf pulls in output.obj from libucrt.lib, and this object file pulls in wcstol from strtox.obj, which then later triggers the duplicate definition.

With libucrt.lib from WinSDK 10.0.22621.0, this no longer happens. But the test still failed, fulfilling the XFAIL, with the following error:

AddressSanitizer: CHECK failed: asan_rtl.cpp:387 "((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, 0x0) (tid=16328)
    <empty stack>

After this patch, this no longer triggers errors, and the test runs successfully. So the original XFAIL condition is dependent on WinSDK version, but there was a second error which made the XFAIL still seem to behave correctly, until after this change.

I can't comment on the rest of whether it's expected that this error condition that previously aborted execution, now no longer triggers a failure.

Thanks for following up this lead. I don't know how the CHECK is triggered, my guess is when __debugbreak() is executed, some exception handlers in asan gets called with the breakpoint exception causing a nested asan init?