[asan] Use 32TB as shadow offset for asan on Windows64

[wang0109]

Summary

This is part of the effort for asan to support Windows 64 bit.
The large offset is being tested on Windows 10 (which has larger usable
virtual address space than Windows 8 or earlier)

[wang0109]

• update diff: add ULL to end of offsets

[wang0109]

• update diff: use shift notation

[filcab]

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

755

Don't change this. This code is suspicious and seems to have either bit-rotted or still only support Linux.

lib/Transforms/Instrumentation/AddressSanitizer.cpp

82

Since all the other 1-bit-set constants around here use 1ULL << XX, can you switch to 1ULL << 45?

[etienneb]

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

755

Agree, this will break linux instrumentation.

Note: Google Test filter = AddressSanitizer.asm_rw
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from AddressSanitizer
[ RUN      ] AddressSanitizer.asm_rw
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:135: Failure
Death test: asm_write(&buf[1], static_cast<T>(0))
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:135: Failure
Death test: asm_write(&buf[1], static_cast<T>(0))
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:135: Failure
Death test: asm_write(&buf[1], static_cast<T>(0))
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:135: Failure
Death test: asm_write(&buf[1], static_cast<T>(0))
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:149: Failure
Death test: asm_write<__m128i>((__m128i*) p, val)
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:158: Failure
Death test: asm_read(&buf[1])
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:158: Failure
Death test: asm_read(&buf[1])
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:158: Failure
Death test: asm_read(&buf[1])
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:158: Failure
Death test: asm_read(&buf[1])
    Result: failed to die.
 Error msg:
[  DEATH   ] 
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/asan/tests/asan_asm_test.cc:170: Failure
Death test: asm_read<__m128i>((__m128i*) p)
    Result: failed to die.
 Error msg:
[  DEATH   ] 
[  FAILED  ] AddressSanitizer.asm_rw (165 ms)
[----------] 1 test from AddressSanitizer (165 ms total)

[wang0109]

• update diff: guard for windows 64, no impact on linux

[etienneb]

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

754

It's still not valid.
Clang can cross compile, you it must be able to emit any of these two constants, depending on the target.

[filcab]

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

754

This class doesn't get instantiated if the target is not Linux on x86-64. This is also not the way to make it support Windows. You can't use the pre-processor to support it, since we might be cross-compiling.
Please don't touch this file yet. Deal with plain ASan first, and afterwards enhance it by enabling ASM instrumentation.

[wang0109]

X86AsmInstrumentation *
CreateX86AsmInstrumentation(const MCTargetOptions &MCOptions,
                            const MCContext &Ctx, const MCSubtargetInfo *&STI) {
  Triple T(STI->getTargetTriple());
  const bool hasCompilerRTSupport = T.isOSLinux();
  if (ClAsanInstrumentAssembly && hasCompilerRTSupport &&
      MCOptions.SanitizeAddress) {
    if (STI->getFeatureBits()[X86::Mode32Bit] != 0)
      return new X86AddressSanitizer32(STI);
    if (STI->getFeatureBits()[X86::Mode64Bit] != 0)
      return new X86AddressSanitizer64(STI);
  }
  return new X86AsmInstrumentation(STI);
}

Looks like, instrumentation of inline asm is only for Linux. On windows, it is not used. Also, windows 64's Microsoft compiler has no support for inline asm anyway.

[wang0109]

Yes, I am reverting the change to X86AsmInstrumentation.

[wang0109]

• update diff: Revert change to 64bit asm instrumentation offset

[rnk]

lgtm

We could have a test in llvm/test/Instrumentation/AddressSanitizer/win64.ll like the freebsd.ll test, but it seems fairly trivial and low value.

lib/Transforms/Instrumentation/AddressSanitizer.cpp

403–404

clang-format would put the 'else' on the line with the previous '}'.

[wang0109]

• update diff: fix formatting

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

InstrumentMOVSBase(X86::EDI /* DstReg */, X86::ESI /* SrcReg */,
                      X86::ECX /* CntReg */, AccessSize, Ctx, Out);
 
   EmitLabel(Out, DoneSym);
   RestoreFlags(Out);
 }
 
 class X86AddressSanitizer64 : public X86AddressSanitizer {
 public:
-  static const long kShadowOffset = 0x7fff8000;
+  // TODO(wwchrome): This value is experimental and may change.
+  static const long kShadowOffset = 0x1ULL << 45; // 32TB
 
   X86AddressSanitizer64(const MCSubtargetInfo *&STI)
       : X86AddressSanitizer(STI) {}
 
   ~X86AddressSanitizer64() override {}
 
   unsigned GetFrameReg(const MCContext &Ctx, MCStreamer &Out) {
     unsigned FrameReg = GetFrameRegGeneric(Ctx, Out);

lib/Transforms/Instrumentation/AddressSanitizer.cpp

 static const uint64_t kPPC64_ShadowOffset64 = 1ULL << 41;
 static const uint64_t kSystemZ_ShadowOffset64 = 1ULL << 52;
 static const uint64_t kMIPS32_ShadowOffset32 = 0x0aaa0000;
 static const uint64_t kMIPS64_ShadowOffset64 = 1ULL << 37;
 static const uint64_t kAArch64_ShadowOffset64 = 1ULL << 36;
 static const uint64_t kFreeBSD_ShadowOffset32 = 1ULL << 30;
 static const uint64_t kFreeBSD_ShadowOffset64 = 1ULL << 46;
 static const uint64_t kWindowsShadowOffset32 = 3ULL << 28;
+// TODO(wwchrome): Experimental for asan Win64, may change.
+static const uint64_t kWindowsShadowOffset64 = 0x1ULL << 45;  // 32TB.
 
 static const size_t kMinStackMallocSize = 1 << 6;   // 64B
 static const size_t kMaxStackMallocSize = 1 << 16;  // 64K
 static const uintptr_t kCurrentStackFrameMagic = 0x41B58AB3;
 static const uintptr_t kRetiredStackFrameMagic = 0x45E0360E;
 
 static const char *const kAsanModuleCtorName = "asan.module_ctor";
 static const char *const kAsanModuleDtorName = "asan.module_dtor";
else if (IsSystemZ)
       Mapping.Offset = kSystemZ_ShadowOffset64;
     else if (IsFreeBSD)
       Mapping.Offset = kFreeBSD_ShadowOffset64;
     else if (IsLinux && IsX86_64) {
       if (IsKasan)
         Mapping.Offset = kLinuxKasan_ShadowOffset64;
       else
         Mapping.Offset = kSmallX86_64ShadowOffset;
-    } else if (IsMIPS64)
+    } else if (IsWindows && IsX86_64) {
+      Mapping.Offset = kWindowsShadowOffset64;
+    }
+    else if (IsMIPS64)
       Mapping.Offset = kMIPS64_ShadowOffset64;
     else if (IsIOS)
       // If we're targeting iOS and x86, the binary is built for iOS simulator.
       Mapping.Offset = IsX86_64 ? kIOSSimShadowOffset64 : kIOSShadowOffset64;
     else if (IsAArch64)
       Mapping.Offset = kAArch64_ShadowOffset64;
     else
       Mapping.Offset = kDefaultShadowOffset64;