[Compiler-rt][Sanitizer][MSan][i386] Adding support for i386

[mohit.bhakkad]

Summary

Most of the changes are specific to i386 architecture to support Memory Sanitizer.

[eugenis]

Are you sure you are testing i386? :)
It would require changes to lib/msan/tests/CMakeLists.txt and something under test/msan, and this CL does not touch those files. I've tried this patch and 32-bit tests are not getting built.

[mohit.bhakkad]

In D6433#5, @eugenis wrote:

Are you sure you are testing i386? :)
It would require changes to lib/msan/tests/CMakeLists.txt and something under test/msan, and this CL does not touch those files. I've tried this patch and 32-bit tests are not getting built.

Yes I am sure. But in test/msan all the tests contain hardcoded -m64.
Currently for the sake of testing, I have replaced all these -m64 with -m32.

[eugenis]

Ah, OK. In ASan tests there is code (in lit.cfg) that adds -m32 or -m64 to %clangxx depending on the test suite configuration, could you copy that to msan and remove hardcoded -m64?

Please don't forget about msan_test.cc.

I'm sure you'll run into issues with va-arg functions: see VarArgAMD64Helper in MemorySanitizer.cpp, it needs to be reimplemented for i386 and probably both MIPS variants. Tests for that are in msan_test.cc.

[mohit.bhakkad]

I am not able to test it using lit, as while building compiler-rt my clang is not supporting i386.
I have posted this issue in mailing list also.
Right now I have used gcc/g++ to build it and tests on this doc are giving correct results, when tested using clang individually.

[mohit.bhakkad]

@eugenis could you please comment on this, as I am looking forward to provide VarArgHelper for i386 too.

[eugenis]

I've replied in the email thread.

[mohit.bhakkad]

Update:

• Thanks @eugenis, issue with i386 support is resolved.

• Right now, while testing msan32 using lit, most of the tests are failing with:

FATAL: Code 0x5556aec0 is out of application range. Non-PIE build?
FATAL: MemorySanitizer can not mmap the shadow memory.
FATAL: Make sure to compile with -fPIE and to link with -pie.
FATAL: Disabling ASLR is known to cause this error.
FATAL: If running under GDB, try 'set disable-randomization off'.
==23092==Process memory map follows:
	0x55555000-0x555ed000	/home/mohit/compiler-rt_build/test/msan/Output/strlen_of_shadow.cc.tmp
	0x555ed000-0x555ee000	/home/mohit/compiler-rt_build/test/msan/Output/strlen_of_shadow.cc.tmp
	0x555ee000-0x555f1000	/home/mohit/compiler-rt_build/test/msan/Output/strlen_of_shadow.cc.tmp
	0x555f1000-0x56648000	
	0x56648000-0x56668000	/lib/i386-linux-gnu/ld-2.19.so
	0x56668000-0x56669000	/lib/i386-linux-gnu/ld-2.19.so
	0x56669000-0x5666a000	/lib/i386-linux-gnu/ld-2.19.so
	0x5666a000-0x5666b000	[vdso]
	0x5666b000-0x56672000	
	0x5668c000-0x56768000	/usr/lib/i386-linux-gnu/libstdc++.so.6.0.19
	0x56768000-0x56769000	/usr/lib/i386-linux-gnu/libstdc++.so.6.0.19
	0x56769000-0x5676d000	/usr/lib/i386-linux-gnu/libstdc++.so.6.0.19
	0x5676d000-0x5676e000	/usr/lib/i386-linux-gnu/libstdc++.so.6.0.19
	0x5676e000-0x56775000	
	0x56775000-0x567b9000	/lib/i386-linux-gnu/libm-2.19.so
	0x567b9000-0x567ba000	/lib/i386-linux-gnu/libm-2.19.so
	0x567ba000-0x567bb000	/lib/i386-linux-gnu/libm-2.19.so
	0x567bb000-0x567bc000	
	0x567bc000-0x567d4000	/lib/i386-linux-gnu/libpthread-2.19.so
	0x567d4000-0x567d5000	/lib/i386-linux-gnu/libpthread-2.19.so
	0x567d5000-0x567d6000	/lib/i386-linux-gnu/libpthread-2.19.so
	0x567d6000-0x567d8000	
	0x567d8000-0x567df000	/lib/i386-linux-gnu/librt-2.19.so
	0x567df000-0x567e0000	/lib/i386-linux-gnu/librt-2.19.so
	0x567e0000-0x567e1000	/lib/i386-linux-gnu/librt-2.19.so
	0x567e1000-0x567e4000	/lib/i386-linux-gnu/libdl-2.19.so
	0x567e4000-0x567e5000	/lib/i386-linux-gnu/libdl-2.19.so
	0x567e5000-0x567e6000	/lib/i386-linux-gnu/libdl-2.19.so
	0x567e6000-0x56802000	/lib/i386-linux-gnu/libgcc_s.so.1
	0x56802000-0x56803000	/lib/i386-linux-gnu/libgcc_s.so.1
	0x56803000-0x569ac000	/lib/i386-linux-gnu/libc-2.19.so
	0x569ac000-0x569ae000	/lib/i386-linux-gnu/libc-2.19.so
	0x569ae000-0x569af000	/lib/i386-linux-gnu/libc-2.19.so
	0x569af000-0x56b5f000	
	0xffac2000-0xffae3000	[stack]
==23092==End of process memory map.

but when I run tests individually, almost all tests are giving expected o/p:
for eg:(see that I have used exact options that of in lit suite)

mohit@mohit:~$ /home/mohit/LLVM_x86/install/bin/clang --driver-mode=g++ -fsanitize=memory -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls  -gline-tables-only -m32 -fsanitize-memory-track-origins=2  -O3 /home/mohit/compiler-rt/test/msan/chained_origin.cc
mohit@mohit:~$ ./a.out 
==23140== WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xf77b17dd in main /home/mohit/compiler-rt/test/msan/chained_origin.cc:47:3
    #1 0xf73e6a82 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x19a82)
    #2 0xf774be95 in _start (/home/mohit/a.out+0x14e95)

  Uninitialized value was stored to memory at
    #0 0xf77b1722 in fn_h() /home/mohit/compiler-rt/test/msan/chained_origin.cc:35:5
    #1 0xf77b17a2 in main /home/mohit/compiler-rt/test/msan/chained_origin.cc:46:3
    #2 0xf73e6a82 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x19a82)

  Uninitialized value was stored to memory at
    #0 0xf77b1692 in fn_g(int) /home/mohit/compiler-rt/test/msan/chained_origin.cc:25:5
    #1 0xf77b16bb in fn_f(int) /home/mohit/compiler-rt/test/msan/chained_origin.cc:30:3
    #2 0xf77b179d in main /home/mohit/compiler-rt/test/msan/chained_origin.cc:45:3
    #3 0xf73e6a82 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x19a82)

  Uninitialized value was created by an allocation of 'z' in the stack frame of function 'main'
    #0 0xf77b1730 in main /home/mohit/compiler-rt/test/msan/chained_origin.cc:38

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/mohit/compiler-rt/test/msan/chained_origin.cc:47 main
Exiting

I am pretty sure about the correctness of address ranges in this patch. It looks like it is failing some check, investigating it now.

tl;dr: most of the test cases are passing individually, but with lit test suite, almost all are failing due to conflict in addresses.

[eugenis]

0x555 is where a PIE binary is mapped when ASLR is disabled (1/3rd of
the address space). No idea why that happens with lit.

cmake/config-ix.cmake

 filter_available_targets(ASAN_SUPPORTED_ARCH
   x86_64 i386 i686 powerpc64 powerpc64le arm mips mipsel mips64 mips64el)
 filter_available_targets(DFSAN_SUPPORTED_ARCH x86_64 mips64 mips64el)
 filter_available_targets(LSAN_SUPPORTED_ARCH x86_64 mips64 mips64el)
 # LSan common files should be available on all architectures supported
 # by other sanitizers (even if they build into dummy object files).
 filter_available_targets(LSAN_COMMON_SUPPORTED_ARCH
   ${SANITIZER_COMMON_SUPPORTED_ARCH})
-filter_available_targets(MSAN_SUPPORTED_ARCH x86_64 mips64 mips64el)
+filter_available_targets(MSAN_SUPPORTED_ARCH i386 x86_64 mips64 mips64el)
 filter_available_targets(PROFILE_SUPPORTED_ARCH x86_64 i386 i686 arm mips mips64
   mipsel mips64el aarch64 powerpc64 powerpc64le)
 filter_available_targets(TSAN_SUPPORTED_ARCH x86_64)
 filter_available_targets(UBSAN_SUPPORTED_ARCH x86_64 i386 i686 arm aarch64 mips mipsel mips64 mips64el)
 
 if(ANDROID)
   set(OS_NAME "Android")
 else()

lib/msan/msan.h

const MappingDesc kMemoryLayout[] = {
     {0x000000000000ULL, 0x00a000000000ULL, MappingDesc::INVALID, "invalid"},
     {0x00a000000000ULL, 0x00c000000000ULL, MappingDesc::SHADOW, "shadow"},
     {0x00c000000000ULL, 0x00e000000000ULL, MappingDesc::ORIGIN, "origin"},
     {0x00e000000000ULL, 0x010000000000ULL, MappingDesc::APP, "app"}};
 
 #define MEM_TO_SHADOW(mem) (((uptr)(mem)) & ~0x4000000000ULL)
 #define SHADOW_TO_ORIGIN(shadow) (((uptr)(shadow)) + 0x002000000000)
 
+#elif SANITIZER_LINUX && SANITIZER_WORDSIZE == 32
+
+const MappingDesc kMemoryLayout[] = {
+    {0x00000000UL, 0x40000000UL, MappingDesc::INVALID, "invalid"},
+    {0x40000000UL, 0x80000000UL, MappingDesc::SHADOW, "shadow"},
+    {0x80000000UL, 0xc0000000UL, MappingDesc::ORIGIN, "origin"},
+    {0xc0000000UL, 0xffffffffUL, MappingDesc::APP, "app"}};
+
+#define MEM_TO_SHADOW(mem) (((uptr)(mem)) & ~0x80000000UL)
+#define SHADOW_TO_ORIGIN(mem) (((uptr)(mem)) + 0x40000000UL)
+
 #elif SANITIZER_FREEBSD && SANITIZER_WORDSIZE == 64
 
 // Low memory: main binary, MAP_32BIT mappings and modules
 // High memory: heap, modules and main thread stack
 const MappingDesc kMemoryLayout[] = {
     {0x000000000000ULL, 0x010000000000ULL, MappingDesc::APP, "low memory"},
     {0x010000000000ULL, 0x100000000000ULL, MappingDesc::INVALID, "invalid"},
     {0x100000000000ULL, 0x310000000000ULL, MappingDesc::SHADOW, "shadow"},

lib/msan/msan_allocator.cc

void OnUnmap(uptr p, uptr size) const {
     // We are about to unmap a chunk of user memory.
     // Mark the corresponding shadow memory as not needed.
     FlushUnneededShadowMemory(MEM_TO_SHADOW(p), size);
     if (__msan_get_track_origins())
       FlushUnneededShadowMemory(MEM_TO_ORIGIN(p), size);
   }
 };
 
-#if defined(__mips64)
+#if defined (__i386__)
+  static const uptr kMaxAllowedMallocSize = 3UL << 30;
+  static const uptr kRegionSizeLog = 20;
+  static const uptr kNumRegions = SANITIZER_MMAP_RANGE_SIZE >> kRegionSizeLog;
+  typedef FlatByteMap<kNumRegions> ByteMap;
+  typedef CompactSizeClassMap SizeClassMap;
+  typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE, sizeof(Metadata),
+                               SizeClassMap, kRegionSizeLog, ByteMap,
+                               MsanMapUnmapCallback> PrimaryAllocator;
+#elif defined(__mips64)
   static const uptr kMaxAllowedMallocSize = 2UL << 30;
   static const uptr kRegionSizeLog = 20;
   static const uptr kNumRegions = SANITIZER_MMAP_RANGE_SIZE >> kRegionSizeLog;
   typedef TwoLevelByteMap<(kNumRegions >> 12), 1 << 12> ByteMap;
   typedef CompactSizeClassMap SizeClassMap;
 
   typedef SizeClassAllocator32<0, SANITIZER_MMAP_RANGE_SIZE, sizeof(Metadata),
                                SizeClassMap, kRegionSizeLog, ByteMap,

lib/msan/msan_linux.cc

static void CheckMemoryLayoutSanity() {
   uptr prev_end = 0;
   for (unsigned i = 0; i < kMemoryLayoutSize; ++i) {
     uptr start = kMemoryLayout[i].start;
     uptr end = kMemoryLayout[i].end;
     MappingDesc::Type type = kMemoryLayout[i].type;
     CHECK_LT(start, end);
     CHECK_EQ(prev_end, start);
     CHECK(addr_is_type(start, type));
+#if !defined(__i386__)
     CHECK(addr_is_type((start + end) / 2, type));
+#endif
     CHECK(addr_is_type(end - 1, type));
     if (type == MappingDesc::APP) {
       uptr addr = start;
       CHECK(MEM_IS_SHADOW(MEM_TO_SHADOW(addr)));
       CHECK(MEM_IS_ORIGIN(MEM_TO_ORIGIN(addr)));
       CHECK_EQ(MEM_TO_ORIGIN(addr), SHADOW_TO_ORIGIN(MEM_TO_SHADOW(addr)));
 
       addr = (start + end) / 2;

lib/sanitizer_common/sanitizer_platform_interceptors.h

 #define SANITIZER_INTERCEPT_STATVFS SI_FREEBSD || SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_STATVFS64 SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_INITGROUPS SI_NOT_WINDOWS
 #define SANITIZER_INTERCEPT_ETHER_NTOA_ATON SI_NOT_WINDOWS
 #define SANITIZER_INTERCEPT_ETHER_HOST \
   SI_FREEBSD || SI_MAC || SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_ETHER_R SI_FREEBSD || SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_SHMCTL \
-  ((SI_FREEBSD || SI_LINUX_NOT_ANDROID) && SANITIZER_WORDSIZE == 64)
+  ((SI_FREEBSD || SI_LINUX_NOT_ANDROID) && \
+   (SANITIZER_WORDSIZE == 64 || defined(__i386__)))
 #define SANITIZER_INTERCEPT_RANDOM_R SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_PTHREAD_ATTR_GET SI_NOT_WINDOWS
 #define SANITIZER_INTERCEPT_PTHREAD_ATTR_GETINHERITSCHED \
   SI_FREEBSD || SI_MAC || SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_PTHREAD_ATTR_GETAFFINITY_NP SI_LINUX_NOT_ANDROID
 #define SANITIZER_INTERCEPT_PTHREAD_MUTEXATTR_GETPSHARED SI_NOT_WINDOWS
 #define SANITIZER_INTERCEPT_PTHREAD_MUTEXATTR_GETTYPE SI_NOT_WINDOWS
 #define SANITIZER_INTERCEPT_PTHREAD_MUTEXATTR_GETPROTOCOL \