Page MenuHomePhabricator

[Sanitizers] Cleanup handling of stat64/statfs64
ClosedPublic

Authored by wrotki on Jun 8 2022, 1:45 PM.

Details

Summary

There are differences in handling of stat64/statfs64 calls by sanitizers between Linux and macOS. Versions of macOS starting with 10.6 drop the stat64/statfs64 APIs completely, relying on the linker to redirect stat/statfs to the appropriate 64 bit versions. Emitting variables needed by sanitizers is thus controlled by convoluted sets of conditions, involving Linux, IOS, macOS and Android, sprinkled around files.

This change adresses it, allowing to specify presence/absence of stat64/statfs64 for each platform, in a single location. Also, it adresses the Android case which handles stat64, but not statfs64.

Adding Vitaly as a reviewer since he seems to be actively working on sanitizers, perhaps can comment on the Android bit

Diff Detail

Event Timeline

wrotki created this revision.Jun 8 2022, 1:45 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 8 2022, 1:45 PM
Herald added a subscriber: Enna1. · View Herald Transcript
wrotki requested review of this revision.Jun 8 2022, 1:45 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 8 2022, 1:45 PM
Herald added a subscriber: Restricted Project. · View Herald Transcript
yln added a subscriber: eugenis.Jun 8 2022, 4:39 PM

LGTM, thanks!

@vitalybuka @eugenis: any concerns?

yln accepted this revision.Jun 8 2022, 4:39 PM
This revision is now accepted and ready to land.Jun 8 2022, 4:39 PM
vitalybuka accepted this revision.Jun 8 2022, 4:56 PM
This revision was landed with ongoing or failed builds.Jun 9 2022, 12:51 PM
This revision was automatically updated to reflect the committed changes.
dyung added a subscriber: dyung.Jun 10 2022, 1:09 AM

This seems to be causing 7 test failures on our internal linux buildbot:

  • MemorySanitizer-X86_64 :: Linux/glob_altdirfunc.cpp
  • MemorySanitizer-X86_64 :: stat64.cpp
  • MemorySanitizer-lld-X86_64 :: Linux/glob_altdirfunc.cpp
  • MemorySanitizer-lld-X86_64 :: stat64.cpp
  • SanitizerCommon-asan-x86_64-Linux :: Posix/lstat64.cpp
  • SanitizerCommon-msan-x86_64-Linux :: Posix/lstat64.cpp
  • SanitizerCommon-tsan-x86_64-Linux :: Posix/lstat64.cpp

The failures are all similar and look like this:

FAIL: MemorySanitizer-X86_64 :: Linux/glob_altdirfunc.cpp (2151 of 68143)
******************** TEST 'MemorySanitizer-X86_64 :: Linux/glob_altdirfunc.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';      /home/siadmin/jenkins/w/opensource/opensource_build/build/./bin/clang  --driver-mode=g++ -fsanitize=memory -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls  -m64  -gline-tables-only -O0 /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp -o /home/siadmin/jenkins/w/opensource/opensource_build/build/projects/compiler-rt/test/msan/X86_64/Linux/Output/glob_altdirfunc.cpp.tmp &&  /home/siadmin/jenkins/w/opensource/opensource_build/build/projects/compiler-rt/test/msan/X86_64/Linux/Output/glob_altdirfunc.cpp.tmp /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux 2>&1 | FileCheck /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp
: 'RUN: at line 2';      /home/siadmin/jenkins/w/opensource/opensource_build/build/./bin/clang  --driver-mode=g++ -fsanitize=memory -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls  -m64  -gline-tables-only -O0 -D_FILE_OFFSET_BITS=64 /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp -o /home/siadmin/jenkins/w/opensource/opensource_build/build/projects/compiler-rt/test/msan/X86_64/Linux/Output/glob_altdirfunc.cpp.tmp &&  /home/siadmin/jenkins/w/opensource/opensource_build/build/projects/compiler-rt/test/msan/X86_64/Linux/Output/glob_altdirfunc.cpp.tmp /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux 2>&1 | FileCheck /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp
: 'RUN: at line 3';      /home/siadmin/jenkins/w/opensource/opensource_build/build/./bin/clang  --driver-mode=g++ -fsanitize=memory -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls  -m64  -gline-tables-only -O3 /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp -o /home/siadmin/jenkins/w/opensource/opensource_build/build/projects/compiler-rt/test/msan/X86_64/Linux/Output/glob_altdirfunc.cpp.tmp &&  /home/siadmin/jenkins/w/opensource/opensource_build/build/projects/compiler-rt/test/msan/X86_64/Linux/Output/glob_altdirfunc.cpp.tmp /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux 2>&1 | FileCheck /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp
--
Exit Code: 1

Command Output (stderr):
--
/home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp:76:12: error: CHECK: expected string not found in input
 // CHECK: PASS
           ^
<stdin>:1:1: note: scanning from here
MemorySanitizer:DEADLYSIGNAL
^
<stdin>:1:4: note: possible intended match here
MemorySanitizer:DEADLYSIGNAL
   ^

Input file: <stdin>
Check file: /home/siadmin/jenkins/w/opensource/opensource_build/working_dir/compiler-rt/test/msan/Linux/glob_altdirfunc.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            1: MemorySanitizer:DEADLYSIGNAL 
check:76'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:76'1        ?                          possible intended match
            2: ==24041==ERROR: MemorySanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 bp 0x7ffc037a9bc0 sp 0x7ffc037a9b98 T24041) 
check:76'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            3: ==24041==Hint: pc points to the zero page. 
check:76'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            4: ==24041==The signal is caused by a READ memory access. 
check:76'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            5: ==24041==Hint: address points to the zero page. 
check:76'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            6:  #0 0x0 (<unknown module>) 
check:76'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
            .
            .
            .
>>>>>>

--

********************

Can you take a look and revert if you need time to investigate?

n-omer added a subscriber: n-omer.Jun 10 2022, 4:48 AM

Okay, will revert and investigate - thanks for notifying.

hctim added a subscriber: hctim.Jun 10 2022, 1:50 PM

(jumping on the bandwagon, also failing on the sanitizer-powerpc bot: https://lab.llvm.org/buildbot/#/builders/105/builds/26531. Maybe this is easier to diagnose than internal bots.)