[HWASan] Use page aliasing on x86_64.

[morehouse]

Summary

Userspace page aliasing allows us to use middle pointer bits for tags
without untagging them before syscalls or accesses. This should enable
easier experimentation with HWASan on x86_64 platforms.

Currently stack, global, and secondary heap tagging are unsupported.
Only primary heap allocations get tagged.

Note that aliasing mode will not work properly in the presence of
fork(), since heap memory will be shared between the parent and child
processes. This mode is non-ideal; we expect Intel LAM to enable full
HWASan support on x86_64 in the future.

[eugenis]

LGTM

compiler-rt/lib/hwasan/hwasan_dynamic_shadow.cpp

125

this seems kind of arbitrary, why not RingBufferSize()?

[morehouse]

• Fix includes; use RingBufferSize().

[morehouse]

Removing all the UNTAGs from tests caused an aarch64 bot (presumably without TBI) to fail: https://lab.llvm.org/buildbot/#/builders/53/builds/1634

I'll put the UNTAGs back in but make them no-ops on x86.

[eugenis]

AArch64 had TBI from the start, but the kernel support was added only recently. Add untags only in the failing tests, for memory that is passed to system calls.

[morehouse]

• Add back UNTAGs for Linux syscalls.

[EccoTheDolphin]

compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp

11–14

@morehouse and @eugenis it seems that __hwasan_tag_memory expects untagged pointer. This particular piece looks a little bit suspicious. Could you please clarify?

[morehouse]

compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp

11–14

Nice catch. I think it currently works since allocator tagging is disabled by default for the HWASan tests.

But we probably shouldn't rely on that.

[morehouse]

compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp

11–14

We do need the malloc instead of mmap on x86 since only the heap is aliased.

Submitted https://reviews.llvm.org/rG3511022f5f0a to explicitly untag the pointer.

[xiangzhangllvm]

compiler-rt/lib/hwasan/hwasan.h

41

Hello @morehouse, I have some questions here:

1. Does the "page alias" in your patch mean "tagged address and untagged address map to same place" ?
2. "The alias region is placed next to the shadow", what does the "alias region" here mean?
3. Why kAddressTagShift = 39 , kTagBits = 3 ? Does it mean the "alias region" size is 2^39Bytes ? It seems to me that the ""alias region is placed inside the shadow?

Thank you !

[morehouse]

compiler-rt/lib/hwasan/hwasan.h

41

Hello @morehouse, I have some questions here:

1. Does the "page alias" in your patch mean "tagged address and untagged address map to same place" ?

Yes, the virtual addresses map to the same physical addresses.

2. "The alias region is placed next to the shadow", what does the "alias region" here mean?

We preallocate address space for the entire heap and create aliases for that region at startup. So "alias region" == heap and aliases.

3. Why kAddressTagShift = 39 , kTagBits = 3 ?

kTagBits = 3 is due to some preliminary performance benchmarks. Anything higher than that seems to greatly increase overhead. kAddressTagShift = 39 is to allow a contiguous heap of 512GB, which should be plenty in practice.

Does it mean the "alias region" size is 2^39Bytes ?

Heap is 2^39 == 512GB. Then we have 7 other aliases of that memory, for a total alias region size of 2^42 == 4TB.

It seems to me that the ""alias region is placed inside the shadow?

The shadow region on x86_64 is 8TB starting at a dynamic offset. We dynamically place the alias region (heap and aliases) immediately following the shadow region.

Thank you !

[xiangzhangllvm]

compiler-rt/lib/hwasan/hwasan.h

41

Thank you!
I think I understand:
We "pre-allocate" 4TB for heap, but we just really used in 512GB, so the bit 39-42 can be use as tag for a heap point!

compiler-rt/lib/hwasan/hwasan.h

 //===-- hwasan.h ------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of HWAddressSanitizer.
 //
 // Private Hwasan header.
 //===----------------------------------------------------------------------===//
 
 #ifndef HWASAN_H
 #define HWASAN_H
 
+#include "hwasan_flags.h"
+#include "hwasan_interface_internal.h"
+#include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_flags.h"
 #include "sanitizer_common/sanitizer_internal_defs.h"
 #include "sanitizer_common/sanitizer_stacktrace.h"
-#include "hwasan_interface_internal.h"
-#include "hwasan_flags.h"
 #include "ubsan/ubsan_platform.h"
 
 #ifndef HWASAN_CONTAINS_UBSAN
 # define HWASAN_CONTAINS_UBSAN CAN_SANITIZE_UB
 #endif
 
 #ifndef HWASAN_WITH_INTERCEPTORS
 #define HWASAN_WITH_INTERCEPTORS 0
 #endif
 
 #ifndef HWASAN_REPLACE_OPERATORS_NEW_AND_DELETE
 #define HWASAN_REPLACE_OPERATORS_NEW_AND_DELETE HWASAN_WITH_INTERCEPTORS
 #endif
 
 typedef u8 tag_t;
 
+#if defined(__x86_64__)
+// Tags are done in middle bits using userspace aliasing.
+constexpr unsigned kAddressTagShift = 39;
+constexpr unsigned kTagBits = 3;
+
+// The alias region is placed next to the shadow so the upper bits of all
+// taggable addresses matches the upper bits of the shadow base.  This shift
+// value determines which upper bits must match.  It has a floor of 44 since the
+// shadow is always 8TB.
+// TODO(morehouse): In alias mode we can shrink the shadow and use a
+// simpler/faster shadow calculation.
+constexpr unsigned kTaggableRegionCheckShift =
+    __sanitizer::Max(kAddressTagShift + kTagBits + 1U, 44U);
+#else
 // TBI (Top Byte Ignore) feature of AArch64: bits [63:56] are ignored in address
 // translation and can be used to store a tag.
 constexpr unsigned kAddressTagShift = 56;
 constexpr unsigned kTagBits = 8;
+#endif  // defined(__x86_64__)
 
 // Mask for extracting tag bits from the lower 8 bits.
 constexpr uptr kTagMask = (1UL << kTagBits) - 1;
 
-// Masks for extracting and removing tags from full pointers.
+// Mask for extracting tag bits from full pointers.
 constexpr uptr kAddressTagMask = kTagMask << kAddressTagShift;
 
 // Minimal alignment of the shadow base address. Determines the space available
 // for threads and stack histories. This is an ABI constant.
 const unsigned kShadowBaseAlignment = 32;
 
 const unsigned kRecordAddrBaseTagShift = 3;
 const unsigned kRecordFPShift = 48;

compiler-rt/lib/hwasan/hwasan_allocator.h

 //===-- hwasan_allocator.h --------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of HWAddressSanitizer.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef HWASAN_ALLOCATOR_H
 #define HWASAN_ALLOCATOR_H
 
+#include "hwasan.h"
+#include "hwasan_interface_internal.h"
 #include "hwasan_poisoning.h"
 #include "sanitizer_common/sanitizer_allocator.h"
 #include "sanitizer_common/sanitizer_allocator_checks.h"
 #include "sanitizer_common/sanitizer_allocator_interface.h"
 #include "sanitizer_common/sanitizer_allocator_report.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "sanitizer_common/sanitizer_ring_buffer.h"
 
void OnUnmap(uptr p, uptr size) const {
     TagMemory(p, size, 0);
   }
 };
 
 static const uptr kMaxAllowedMallocSize = 1UL << 40;  // 1T
 
 struct AP64 {
   static const uptr kSpaceBeg = ~0ULL;
+
+#if defined(__x86_64__)
+  static const uptr kSpaceSize = 1ULL << kAddressTagShift;
+#else
   static const uptr kSpaceSize = 0x2000000000ULL;
+#endif
   static const uptr kMetadataSize = sizeof(Metadata);
   typedef __sanitizer::VeryDenseSizeClassMap SizeClassMap;
   using AddressSpaceView = LocalAddressSpaceView;
   typedef HwasanMapUnmapCallback MapUnmapCallback;
   static const uptr kFlags = 0;
 };
 typedef SizeClassAllocator64<AP64> PrimaryAllocator;
 typedef CombinedAllocator<PrimaryAllocator> Allocator;
struct HeapAllocationRecord {
   u32  requested_size;
 };
 
 typedef RingBuffer<HeapAllocationRecord> HeapAllocationsRingBuffer;
 
 void GetAllocatorStats(AllocatorStatCounters s);
 
 inline bool InTaggableRegion(uptr addr) {
-  // TODO: specialize for x86 once we use aliasing mode in the allocator.
+#if defined(__x86_64__)
+  // Aliases are mapped next to shadow so that the upper bits match the shadow
+  // base.
+  return (addr >> kTaggableRegionCheckShift) ==
+         (__hwasan_shadow_memory_dynamic_address >> kTaggableRegionCheckShift);
+#endif
   return true;
 }
 
 } // namespace __hwasan
 
 #endif // HWASAN_ALLOCATOR_H

compiler-rt/lib/hwasan/hwasan_allocator.cpp

 void GetAllocatorStats(AllocatorStatCounters s) {
   allocator.GetStats(s);
 }
 
 void HwasanAllocatorInit() {
   atomic_store_relaxed(&hwasan_allocator_tagging_enabled,
                        !flags()->disable_allocator_tagging);
   SetAllocatorMayReturnNull(common_flags()->allocator_may_return_null);
-  allocator.Init(common_flags()->allocator_release_to_os_interval_ms);
+  allocator.Init(common_flags()->allocator_release_to_os_interval_ms,
+                 kAliasRegionStart);
   for (uptr i = 0; i < sizeof(tail_magic); i++)
     tail_magic[i] = GetCurrentThread()->GenerateRandomTag();
 }
 
 void AllocatorSwallowThreadLocalCache(AllocatorCache *cache) {
   allocator.SwallowCache(cache);
 }
 
if (AllocatorMayReturnNull())
       return errno_EINVAL;
     ReportInvalidPosixMemalignAlignment(alignment, stack);
   }
   void *ptr = HwasanAllocate(stack, size, alignment, false);
   if (UNLIKELY(!ptr))
     // OOM error is already taken care of by HwasanAllocate.
     return errno_ENOMEM;
   CHECK(IsAligned((uptr)ptr, alignment));
-  *(void **)UntagPtr(memptr) = ptr;
+  *memptr = ptr;
   return 0;
 }
 
 void hwasan_free(void *ptr, StackTrace *stack) {
   return HwasanDeallocate(stack, ptr);
 }
 
 }  // namespace __hwasan

compiler-rt/lib/hwasan/hwasan_dynamic_shadow.cpp

 //===-- hwasan_dynamic_shadow.cpp -------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file is a part of HWAddressSanitizer. It reserves dynamic shadow memory
 /// region and handles ifunc resolver case, when necessary.
 ///
 //===----------------------------------------------------------------------===//
 
-#include "hwasan.h"
 #include "hwasan_dynamic_shadow.h"
-#include "hwasan_mapping.h"
-#include "sanitizer_common/sanitizer_common.h"
-#include "sanitizer_common/sanitizer_posix.h"
 
 #include <elf.h>
 #include <link.h>
 
+#include "hwasan.h"
+#include "hwasan_mapping.h"
+#include "hwasan_thread_list.h"
+#include "sanitizer_common/sanitizer_common.h"
+#include "sanitizer_common/sanitizer_posix.h"
+
 // The code in this file needs to run in an unrelocated binary. It should not
 // access any external symbol, including its own non-hidden globals.
 
 #if SANITIZER_ANDROID
 extern "C" {
 
 INTERFACE_ATTRIBUTE void __hwasan_shadow();
 decltype(__hwasan_shadow)* __hwasan_premap_shadow();

 
 }  // namespace __hwasan
 #else
 namespace __hwasan {
 
 void InitShadowGOT() {}
 
 uptr FindDynamicShadowStart(uptr shadow_size_bytes) {
+#if defined(__x86_64__)
+  constexpr uptr kAliasSize = 1ULL << kAddressTagShift;
+  constexpr uptr kNumAliases = 1ULL << kTagBits;
+  return MapDynamicShadowAndAliases(shadow_size_bytes, kAliasSize, kNumAliases,
+                                    RingBufferSize());
+#endif
   return MapDynamicShadow(shadow_size_bytes, kShadowScale, kShadowBaseAlignment,
                           kHighMemEnd);
 }
 
 }  // namespace __hwasan
 
 #endif  // SANITIZER_ANDROID

compiler-rt/lib/hwasan/hwasan_flags.h

 //===-- hwasan_flags.h ------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of HWAddressSanitizer.
 //
 //===----------------------------------------------------------------------===//
 #ifndef HWASAN_FLAGS_H
 #define HWASAN_FLAGS_H
 
+#include "sanitizer_common/sanitizer_internal_defs.h"
+
 namespace __hwasan {
 
 struct Flags {
 #define HWASAN_FLAG(Type, Name, DefaultValue, Description) Type Name;
 #include "hwasan_flags.inc"
 #undef HWASAN_FLAG
 
   void SetDefaults();
 };
 
 Flags *flags();
 
 }  // namespace __hwasan
 
 #endif  // HWASAN_FLAGS_H

compiler-rt/lib/hwasan/hwasan_interceptors.cpp

 }
 
 INTERCEPTOR(int, pthread_create, void *th, void *attr, void *(*callback)(void*),
             void * param) {
   ScopedTaggingDisabler disabler;
   ThreadStartArg *A = reinterpret_cast<ThreadStartArg *> (MmapOrDie(
       GetPageSizeCached(), "pthread_create"));
   *A = {callback, param};
-  int res = REAL(pthread_create)(UntagPtr(th), UntagPtr(attr),
-                                 &HwasanThreadStartFunc, A);
+  int res = REAL(pthread_create)(th, attr, &HwasanThreadStartFunc, A);
   return res;
 }
 
 DEFINE_REAL(int, vfork)
 DECLARE_EXTERN_INTERCEPTOR_AND_WRAPPER(int, vfork)
 #endif // HWASAN_WITH_INTERCEPTORS
 
 #if HWASAN_WITH_INTERCEPTORS && defined(__aarch64__)

compiler-rt/lib/hwasan/hwasan_linux.cpp

 uptr kLowMemEnd;
 uptr kLowShadowEnd;
 uptr kLowShadowStart;
 uptr kHighShadowStart;
 uptr kHighShadowEnd;
 uptr kHighMemStart;
 uptr kHighMemEnd;
 
+uptr kAliasRegionStart;  // Always 0 on non-x86.
+
 static void PrintRange(uptr start, uptr end, const char *name) {
   Printf("|| [%p, %p] || %.*s ||\n", (void *)start, (void *)end, 10, name);
 }
 
 static void PrintAddressSpaceLayout() {
   PrintRange(kHighMemStart, kHighMemEnd, "HighMem");
   if (kHighShadowEnd + 1 < kHighMemStart)
     PrintRange(kHighShadowEnd + 1, kHighMemStart - 1, "ShadowGap");

 #define PR_SET_TAGGED_ADDR_CTRL 55
 #define PR_GET_TAGGED_ADDR_CTRL 56
 #define PR_TAGGED_ADDR_ENABLE (1UL << 0)
   // Check we're running on a kernel that can use the tagged address ABI.
   int local_errno = 0;
   if (internal_iserror(internal_prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0),
                        &local_errno) &&
       local_errno == EINVAL) {
-#if SANITIZER_ANDROID
+#if SANITIZER_ANDROID || defined(__x86_64__)
     // Some older Android kernels have the tagged pointer ABI on
     // unconditionally, and hence don't have the tagged-addr prctl while still
     // allow the ABI.
     // If targeting Android and the prctl is not around we assume this is the
     // case.
     return;
 #else
     if (flags()->fail_without_syscall_abi) {
bool InitShadow() {
   // High shadow takes whatever memory is left up there (making sure it is not
   // interfering with low memory in the fixed case).
   kHighShadowEnd = MemToShadow(kHighMemEnd);
   kHighShadowStart = Max(kLowMemEnd, MemToShadow(kHighShadowEnd)) + 1;
 
   // High memory starts where allocated shadow allows.
   kHighMemStart = ShadowToMem(kHighShadowStart);
 
+#if defined(__x86_64__)
+  constexpr uptr kAliasRegionOffset = 1ULL << (kTaggableRegionCheckShift - 1);
+  kAliasRegionStart =
+      __hwasan_shadow_memory_dynamic_address + kAliasRegionOffset;
+
+  CHECK_EQ(kAliasRegionStart >> kTaggableRegionCheckShift,
+           __hwasan_shadow_memory_dynamic_address >> kTaggableRegionCheckShift);
+  CHECK_EQ(
+      (kAliasRegionStart + kAliasRegionOffset - 1) >> kTaggableRegionCheckShift,
+      __hwasan_shadow_memory_dynamic_address >> kTaggableRegionCheckShift);
+#endif
+
   // Check the sanity of the defined memory ranges (there might be gaps).
   CHECK_EQ(kHighMemStart % GetMmapGranularity(), 0);
   CHECK_GT(kHighMemStart, kHighShadowEnd);
   CHECK_GT(kHighShadowEnd, kHighShadowStart);
   CHECK_GT(kHighShadowStart, kLowMemEnd);
   CHECK_GT(kLowMemEnd, kLowMemStart);
   CHECK_GT(kLowShadowEnd, kLowShadowStart);
   CHECK_GT(kLowShadowStart, kLowMemEnd);

compiler-rt/lib/hwasan/hwasan_mapping.h

 extern uptr kLowMemEnd;
 extern uptr kLowShadowEnd;
 extern uptr kLowShadowStart;
 extern uptr kHighShadowStart;
 extern uptr kHighShadowEnd;
 extern uptr kHighMemStart;
 extern uptr kHighMemEnd;
 
+extern uptr kAliasRegionStart;
+
 inline uptr MemToShadow(uptr untagged_addr) {
   return (untagged_addr >> kShadowScale) +
          __hwasan_shadow_memory_dynamic_address;
 }
 inline uptr ShadowToMem(uptr shadow_addr) {
   return (shadow_addr - __hwasan_shadow_memory_dynamic_address) << kShadowScale;
 }
 inline uptr MemToShadowSize(uptr size) {
   return size >> kShadowScale;
 }
 
 bool MemIsApp(uptr p);
 
 }  // namespace __hwasan
 
 #endif  // HWASAN_MAPPING_H

compiler-rt/lib/hwasan/hwasan_memintrinsics.cpp

 #include "hwasan_interface_internal.h"
 #include "sanitizer_common/sanitizer_libc.h"
 
 using namespace __hwasan;
 
 void *__hwasan_memset(void *block, int c, uptr size) {
   CheckAddressSized<ErrorAction::Recover, AccessType::Store>(
       reinterpret_cast<uptr>(block), size);
-  return memset(UntagPtr(block), c, size);
+  return memset(block, c, size);
 }
 
 void *__hwasan_memcpy(void *to, const void *from, uptr size) {
   CheckAddressSized<ErrorAction::Recover, AccessType::Store>(
       reinterpret_cast<uptr>(to), size);
   CheckAddressSized<ErrorAction::Recover, AccessType::Load>(
       reinterpret_cast<uptr>(from), size);
-  return memcpy(UntagPtr(to), UntagPtr(from), size);
+  return memcpy(to, from, size);
 }
 
 void *__hwasan_memmove(void *to, const void *from, uptr size) {
   CheckAddressSized<ErrorAction::Recover, AccessType::Store>(
       reinterpret_cast<uptr>(to), size);
   CheckAddressSized<ErrorAction::Recover, AccessType::Load>(
       reinterpret_cast<uptr>(from), size);
   return memmove(UntagPtr(to), UntagPtr(from), size);
 }

compiler-rt/lib/sanitizer_common/sanitizer_common.h

 
 inline uptr Log2(uptr x) {
   CHECK(IsPowerOfTwo(x));
   return LeastSignificantSetBitIndex(x);
 }
 
 // Don't use std::min, std::max or std::swap, to minimize dependency
 // on libstdc++.
-template<class T> T Min(T a, T b) { return a < b ? a : b; }
-template<class T> T Max(T a, T b) { return a > b ? a : b; }
+template <class T>
+constexpr T Min(T a, T b) {
+  return a < b ? a : b;
+}
+template <class T>
+constexpr T Max(T a, T b) {
+  return a > b ? a : b;
+}
 template<class T> void Swap(T& a, T& b) {
   T tmp = a;
   a = b;
   b = tmp;
 }
 
 // Char handling
 inline bool IsSpace(int c) {

compiler-rt/test/hwasan/TestCases/Linux/aligned_alloc-alignment.cpp

 // RUN: %clangxx_hwasan -O0 %s -o %t
 // RUN: %env_hwasan_opts=allocator_may_return_null=0 not %run %t 2>&1 | FileCheck %s
 // RUN: %env_hwasan_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-NULL
 
 // UNSUPPORTED: android
 
 // REQUIRES: stable-runtime
 
 #include <stdio.h>
 #include <stdlib.h>
 
-#include "../utils.h"
-
 extern void *aligned_alloc(size_t alignment, size_t size);
 
 int main() {
   void *p = aligned_alloc(17, 100);
   // CHECK: ERROR: HWAddressSanitizer: invalid alignment requested in aligned_alloc: 17
   // CHECK: {{#0 0x.* in .*}}{{aligned_alloc|memalign}}
   // CHECK: {{#1 0x.* in main .*aligned_alloc-alignment.cpp:}}[[@LINE-3]]
   // CHECK: SUMMARY: HWAddressSanitizer: invalid-aligned-alloc-alignment
 
-  untag_printf("pointer after failed aligned_alloc: %zd\n", (size_t)p);
+  printf("pointer after failed aligned_alloc: %zd\n", (size_t)p);
   // CHECK-NULL: pointer after failed aligned_alloc: 0
 
   return 0;
 }

compiler-rt/test/hwasan/TestCases/Linux/decorate-proc-maps.c

 // RUN: %clang_hwasan -g %s -o %t
 // RUN: %env_hwasan_opts=decorate_proc_maps=1 %run %t 2>&1 | FileCheck %s --check-prefix=A
 // RUN: %env_hwasan_opts=decorate_proc_maps=1 %run %t 2>&1 | FileCheck %s --check-prefix=B
 
 // A:      rw-p {{.*}}hwasan threads]
 // A-NEXT: ---p {{.*}}shadow gap]
 // A-NEXT: rw-p {{.*}}low shadow]
 // A-NEXT: ---p {{.*}}shadow gap]
 // A-NEXT: rw-p {{.*}}high shadow]
 
-// B-DAG: rw-p {{.*}}SizeClassAllocator: region data]
-// B-DAG: rw-p {{.*}}SizeClassAllocator: region metadata]
-// B-DAG: rw-p {{.*}}SizeClassAllocator: freearray]
 // B-DAG: rw-p {{.*}}SizeClassAllocator: region info]
 // B-DAG: rw-p {{.*}}LargeMmapAllocator]
 // B-DAG: rw-p {{.*}}stack depot]
 
 #include <errno.h>
 #include <fcntl.h>
 #include <pthread.h>
 #include <stdio.h>
 #include <sys/stat.h>
 #include <sys/types.h>
 #include <unistd.h>
 #include <pthread.h>
 #include <stdlib.h>
 
-#include "../utils.h"
+#include "utils.h"
 
 void CopyFdToFd(int in_fd, int out_fd) {
   const size_t kBufSize = 0x10000;
   static char buf[kBufSize];
   while (1) {
     ssize_t got = read(in_fd, UNTAG(buf), kBufSize);
     if (got > 0) {
       write(out_fd, UNTAG(buf), got);
     } else if (got == 0) {
       break;
     } else if (errno != EAGAIN || errno != EWOULDBLOCK || errno != EINTR) {
-      untag_fprintf(stderr, "error reading file, errno %d\n", errno);
+      fprintf(stderr, "error reading file, errno %d\n", errno);
       abort();
     }
   }
 }
 
 void *ThreadFn(void *arg) {
   (void)arg;
   int fd = open(UNTAG("/proc/self/maps"), O_RDONLY);

compiler-rt/test/hwasan/TestCases/Linux/pvalloc-overflow.cpp

 
 #include <assert.h>
 #include <errno.h>
 #include <malloc.h>
 #include <stdint.h>
 #include <string.h>
 #include <unistd.h>
 
-#include "../utils.h"
-
 int main(int argc, char *argv[]) {
   assert(argc == 2);
   const char *action = argv[1];
 
   const size_t page_size = sysconf(_SC_PAGESIZE);
 
   void *p = nullptr;
-  if (!untag_strcmp(action, "m1")) {
+  if (!strcmp(action, "m1")) {
     p = pvalloc((uintptr_t)-1);
-  } else if (!untag_strcmp(action, "psm1")) {
+  } else if (!strcmp(action, "psm1")) {
     p = pvalloc((uintptr_t)-(page_size - 1));
   } else {
     assert(0);
   }
 
-  untag_fprintf(stderr, "errno: %d\n", errno);
+  fprintf(stderr, "errno: %d\n", errno);
 
   return p != nullptr;
 }
 
 // CHECK: {{ERROR: HWAddressSanitizer: pvalloc parameters overflow: size .* rounded up to system page size .* cannot be represented in type size_t}}
 // CHECK: {{#0 0x.* in .*pvalloc}}
 // CHECK: {{#1 0x.* in main .*pvalloc-overflow.cpp:}}
 // CHECK: SUMMARY: HWAddressSanitizer: pvalloc-overflow
 
 // CHECK-NULL: errno: 12

compiler-rt/test/hwasan/TestCases/Linux/release-shadow.c

 // Test that tagging a large region to 0 reduces RSS.
 // RUN: %clang_hwasan -mllvm -hwasan-instrument-stack=0 %s -o %t && %run %t 2>&1
 
 #include <assert.h>
 #include <fcntl.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <sys/mman.h>
 #include <sys/stat.h>
 #include <sys/types.h>
 #include <unistd.h>
 
 #include <sanitizer/hwasan_interface.h>
 
-#include "../utils.h"
+#include "utils.h"
 
 const unsigned char kTag = 42;
 const size_t kNumShadowPages = 256;
 const size_t kNumPages = 16 * kNumShadowPages;
 const size_t kPageSize = 4096;
 const size_t kMapSize = kNumPages * kPageSize;
 
 void sync_rss() {

 size_t current_rss() {
   sync_rss();
   int statm_fd = open(UNTAG("/proc/self/statm"), O_RDONLY);
   assert(statm_fd >= 0);
 
   char buf[100];
   assert(read(statm_fd, &buf, sizeof(buf)) > 0);
   size_t size, rss;
-  assert(sscanf(buf, UNTAG("%zu %zu"), &size, &rss) == 2);
+  assert(sscanf(buf, "%zu %zu", &size, &rss) == 2);
 
   close(statm_fd);
   return rss;
 }
 
 void test_rss_difference(void *p) {
   __hwasan_tag_memory(p, kTag, kMapSize);
   size_t rss_before = current_rss();
   __hwasan_tag_memory(p, 0, kMapSize);
   size_t rss_after = current_rss();
-  untag_fprintf(stderr, "%zu -> %zu\n", rss_before, rss_after);
+  fprintf(stderr, "%zu -> %zu\n", rss_before, rss_after);
   assert(rss_before > rss_after);
   size_t diff = rss_before - rss_after;
-  untag_fprintf(stderr, "diff %zu\n", diff);
+  fprintf(stderr, "diff %zu\n", diff);
   // Check that the difference is at least close to kNumShadowPages.
   assert(diff > kNumShadowPages / 4 * 3);
 }
 
 int main() {
-  untag_fprintf(stderr, "starting rss %zu\n", current_rss());
-  untag_fprintf(stderr, "shadow pages: %zu\n", kNumShadowPages);
+  fprintf(stderr, "starting rss %zu\n", current_rss());
+  fprintf(stderr, "shadow pages: %zu\n", kNumShadowPages);
 
   void *p = mmap(0, kMapSize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
-  untag_fprintf(stderr, "p = %p\n", p);
+  fprintf(stderr, "p = %p\n", p);
 
   test_rss_difference(p);
   test_rss_difference(p);
   test_rss_difference(p);
 
   return 0;
 }

compiler-rt/test/hwasan/TestCases/Linux/reuse-threads.cpp

 // Test that Thread objects are reused.
 // RUN: %clangxx_hwasan -mllvm -hwasan-instrument-stack=0 %s -o %t && %env_hwasan_opts=verbose_threads=1 %run %t 2>&1 | FileCheck %s
 
 #include <assert.h>
 #include <fcntl.h>
 #include <pthread.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
 
 #include <sanitizer/hwasan_interface.h>
 
-#include "../utils.h"
+#include "utils.h"
 
 pthread_barrier_t bar;
 
 void *threadfn(void *) {
   pthread_barrier_wait(UNTAG(&bar));
   return nullptr;
 }
 
void start_stop_threads() {
   for (auto &t : threads)
     pthread_join(t, nullptr);
   pthread_barrier_destroy(UNTAG(&bar));
 }
 
 int main() {
   // Cut off initial threads.
   // CHECK: === test start ===
-  untag_fprintf(stderr, "=== test start ===\n");
+  fprintf(stderr, "=== test start ===\n");
 
   // CHECK: Creating  : T{{[0-9]+}} [[A:0x[0-9a-f]+]] stack:
   // CHECK: Creating  : T{{[0-9]+}} [[B:0x[0-9a-f]+]] stack:
   start_stop_threads();
 
   // CHECK-DAG: Creating  : T{{[0-9]+}} [[A]] stack:
   // CHECK-DAG: Creating  : T{{[0-9]+}} [[B]] stack:
   start_stop_threads();
 
   // CHECK-DAG: Creating  : T{{[0-9]+}} [[A]] stack:
   // CHECK-DAG: Creating  : T{{[0-9]+}} [[B]] stack:
   start_stop_threads();
 
   return 0;
 }

compiler-rt/test/hwasan/TestCases/Linux/utils.h

+#pragma once
+
+#include <stdint.h>
+
+#if defined(__x86_64__)
+#define UNTAG(x) x
+#else
+#define UNTAG(x) (typeof((x) + 0))(((uintptr_t)(x)) & 0xffffffffffffff)
+#endif

compiler-rt/test/hwasan/TestCases/Linux/vfork.c

 // https://github.com/google/sanitizers/issues/925
 // RUN: %clang_hwasan -O0 %s -o %t && %run %t 2>&1
 
 // REQUIRES: aarch64-target-arch || x86_64-target-arch
 
+// Aliasing mode does not support stack tagging.
+// XFAIL: x86_64
+
 #include <assert.h>
 #include <sys/types.h>
 #include <sys/wait.h>
 #include <unistd.h>
 #include <stdio.h>
 #include <sanitizer/hwasan_interface.h>
 
 __attribute__((noinline, no_sanitize("hwaddress"))) void child() {

compiler-rt/test/hwasan/TestCases/Posix/posix_memalign-alignment.cpp

 // RUN: %clangxx_hwasan -O0 %s -o %t
 // RUN: %env_hwasan_opts=allocator_may_return_null=0 not %run %t 2>&1 | FileCheck %s
 // RUN: %env_hwasan_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-NULL
 
 // REQUIRES: stable-runtime
 
 #include <stdio.h>
 #include <stdlib.h>
 
-#include "../utils.h"
-
 int main() {
   void *p = reinterpret_cast<void*>(42);
   int res = posix_memalign(&p, 17, 100);
   // CHECK: ERROR: HWAddressSanitizer: invalid alignment requested in posix_memalign: 17
   // CHECK: {{#0 0x.* in .*posix_memalign}}
   // CHECK: {{#1 0x.* in main .*posix_memalign-alignment.cpp:}}[[@LINE-3]]
   // CHECK: SUMMARY: HWAddressSanitizer: invalid-posix-memalign-alignment
 
-  untag_printf("pointer after failed posix_memalign: %zd\n", (size_t)p);
+  printf("pointer after failed posix_memalign: %zd\n", (size_t)p);
   // CHECK-NULL: pointer after failed posix_memalign: 42
 
   return 0;
 }

compiler-rt/test/hwasan/TestCases/allocator_returns_null.cpp

 #include <assert.h>
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <limits>
 #include <new>
 
-#include "utils.h"
-
 int main(int argc, char **argv) {
   assert(argc == 2);
   const char *action = argv[1];
-  untag_fprintf(stderr, "%s:\n", action);
+  fprintf(stderr, "%s:\n", action);
 
   static const size_t kMaxAllowedMallocSizePlusOne = (1UL << 40) + 1;
 
   void *x = nullptr;
-  if (!untag_strcmp(action, "malloc")) {
+  if (!strcmp(action, "malloc")) {
     x = malloc(kMaxAllowedMallocSizePlusOne);
-  } else if (!untag_strcmp(action, "calloc")) {
+  } else if (!strcmp(action, "calloc")) {
     x = calloc((kMaxAllowedMallocSizePlusOne / 4) + 1, 4);
-  } else if (!untag_strcmp(action, "calloc-overflow")) {
+  } else if (!strcmp(action, "calloc-overflow")) {
     volatile size_t kMaxSizeT = std::numeric_limits<size_t>::max();
     size_t kArraySize = 4096;
     volatile size_t kArraySize2 = kMaxSizeT / kArraySize + 10;
     x = calloc(kArraySize, kArraySize2);
-  } else if (!untag_strcmp(action, "realloc")) {
+  } else if (!strcmp(action, "realloc")) {
     x = realloc(0, kMaxAllowedMallocSizePlusOne);
-  } else if (!untag_strcmp(action, "realloc-after-malloc")) {
+  } else if (!strcmp(action, "realloc-after-malloc")) {
     char *t = (char*)malloc(100);
     *t = 42;
     x = realloc(t, kMaxAllowedMallocSizePlusOne);
     assert(*t == 42);
     free(t);
-  } else if (!untag_strcmp(action, "new")) {
+  } else if (!strcmp(action, "new")) {
     x = operator new(kMaxAllowedMallocSizePlusOne);
-  } else if (!untag_strcmp(action, "new-nothrow")) {
+  } else if (!strcmp(action, "new-nothrow")) {
     x = operator new(kMaxAllowedMallocSizePlusOne, std::nothrow);
   } else {
     assert(0);
   }
 
-  untag_fprintf(stderr, "errno: %d\n", errno);
+  fprintf(stderr, "errno: %d\n", errno);
 
   free(x);
 
   return x != nullptr;
 }
 
 // CHECK-mCRASH: malloc:
 // CHECK-mCRASH: SUMMARY: HWAddressSanitizer: allocation-size-too-big

compiler-rt/test/hwasan/TestCases/heap-buffer-overflow.c

 // RUN: not %run %t 30 20 2>&1 | FileCheck %s --check-prefix=CHECK20
 
 // REQUIRES: stable-runtime
 
 #include <stdlib.h>
 #include <stdio.h>
 #include <sanitizer/hwasan_interface.h>
 
-#include "utils.h"
-
 static volatile char sink;
 
 int main(int argc, char **argv) {
   __hwasan_enable_allocator_tagging();
   int offset = argc < 2 ? 40 : atoi(argv[1]);
   int size = argc < 3 ? 30 : atoi(argv[2]);
   char * volatile x = (char*)malloc(size);
-  untag_fprintf(stderr, "base: %p access: %p\n", x, &x[offset]);
+  fprintf(stderr, "base: %p access: %p\n", x, &x[offset]);
   sink = x[offset];
 
+#if defined(__x86_64__)
+  // Aliasing mode doesn't support the secondary allocator, so we fake a HWASan
+  // report instead of disabling the entire test.
+  if (size == 1000000) {
+    fprintf(stderr, "is a large allocated heap chunk; size: 1003520 offset: %d\n",
+            offset);
+    fprintf(stderr, "is located %s of 1000000-byte region\n",
+            offset == -30 ? "30 bytes to the left" : "0 bytes to the right");
+    return -1;
+  }
+#endif
+
 // CHECK40: allocated heap chunk; size: 32 offset: 8
 // CHECK40: is located 10 bytes to the right of 30-byte region
 //
 // CHECK80: allocated heap chunk; size: 32 offset: 16
 // CHECK80: is located 50 bytes to the right of 30-byte region
 //
 // CHECKm30: is located 30 bytes to the left of 30-byte region
 //

compiler-rt/test/hwasan/TestCases/hwasan-print-shadow.cpp

 // RUN: %clangxx_hwasan -DSIZE=16 -O0 %s -o %t && %run %t 2>&1 | FileCheck %s
 
 // REQUIRES: stable-runtime
 
 #include <assert.h>
 #include <stdlib.h>
 #include <sys/mman.h>
 #include <sanitizer/hwasan_interface.h>
 
 int main() {
-  char *p = (char *)mmap(nullptr, 4096, PROT_READ | PROT_WRITE,
-                         MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
+  char *p = (char *)malloc(4096);
   assert(p);
 
   __hwasan_tag_memory(p, 1, 32);
   __hwasan_tag_memory(p + 32, 3, 16);
   __hwasan_tag_memory(p + 48, 0, 32);
   __hwasan_tag_memory(p + 80, 4, 16);
 
   char *q = (char *)__hwasan_tag_pointer(p, 7);
   __hwasan_print_shadow(q + 5, 89 - 5);
   // CHECK:      HWASan shadow map for {{.*}}5 .. {{.*}}9 (pointer tag 7)
   // CHECK-NEXT:   {{.*}}0: 1
   // CHECK-NEXT:   {{.*}}0: 1
   // CHECK-NEXT:   {{.*}}0: 3
   // CHECK-NEXT:   {{.*}}0: 0
   // CHECK-NEXT:   {{.*}}0: 0
   // CHECK-NEXT:   {{.*}}0: 4
+
+  free(p);
 }

compiler-rt/test/hwasan/TestCases/malloc_fill.cpp

 // Check that we fill malloc-ed memory correctly.
 // RUN: %clangxx_hwasan %s -o %t
 // RUN: %run %t | FileCheck %s --check-prefix=CHECK-0
 // RUN: %env_hwasan_opts=max_malloc_fill_size=20 %run %t | FileCheck %s --check-prefix=CHECK-20-be
 // RUN: %env_hwasan_opts=max_malloc_fill_size=0:malloc_fill_byte=8 %run %t | FileCheck %s --check-prefix=CHECK-0
 // RUN: %env_hwasan_opts=max_malloc_fill_size=10:malloc_fill_byte=8 %run %t | FileCheck %s --check-prefix=CHECK-10-8
 // RUN: %env_hwasan_opts=max_malloc_fill_size=20:malloc_fill_byte=171 %run %t | FileCheck %s --check-prefix=CHECK-20-ab
 
 #include <stdio.h>
 
-#include "utils.h"
-
 int main(int argc, char **argv) {
   // With asan allocator this makes sure we get memory from mmap.
   static const int kSize = 1 << 25;
   unsigned char *x = new unsigned char[kSize];
-  untag_printf("-");
+  printf("-");
   for (int i = 0; i <= 32; i++) {
-    untag_printf("%02x", x[i]);
+    printf("%02x", x[i]);
   }
-  untag_printf("-\n");
+  printf("-\n");
   delete [] x;
 }
 
 // CHECK-0: -000000000000000000000000000000000000000000000000000000000000000000-
 // CHECK-20-be: -bebebebebebebebebebebebebebebebebebebebe00000000000000000000000000-
 // CHECK-10-8: -080808080808080808080000000000000000000000000000000000000000000000-
 // CHECK-20-ab: -abababababababababababababababababababab00000000000000000000000000-

compiler-rt/test/hwasan/TestCases/many-threads-uaf.c

 // RUN: %clang_hwasan %s -o %t && not %env_hwasan_opts=verbose_threads=1 %run %t 2>&1 | FileCheck %s
 // REQUIRES: stable-runtime
 
 #include <pthread.h>
 #include <stdlib.h>
 #include <stdio.h>
 
 #include <sanitizer/hwasan_interface.h>
 
-#include "utils.h"
-
 void *BoringThread(void *arg) {
   char * volatile x = (char*)malloc(10);
   x[5] = 0;
   free(x);
   return NULL;
 }
 
 // CHECK: Creating  : T0
 // CHECK: Creating  : T1
 // CHECK: Destroying: T1
 // CHECK: Creating  : T1100
 // CHECK: Destroying: T1100
 // CHECK: Creating  : T1101
 
 void *UAFThread(void *arg) {
   char * volatile x = (char*)malloc(10);
-  untag_fprintf(stderr, "ZZZ %p\n", x);
+  fprintf(stderr, "ZZZ %p\n", x);
   free(x);
   x[5] = 42;
   // CHECK: ERROR: HWAddressSanitizer: tag-mismatch on address
   // CHECK: WRITE of size 1
   // CHECK: many-threads-uaf.c:[[@LINE-3]]
   // CHECK: Thread: T1101
   return NULL;
 }

compiler-rt/test/hwasan/TestCases/mem-intrinsics.c

 // RUN: %clang_hwasan %s -DTEST_NO=1 -mllvm -hwasan-instrument-mem-intrinsics -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=WRITE
 // RUN: %clang_hwasan %s -DTEST_NO=2 -mllvm -hwasan-instrument-mem-intrinsics -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=READ
 // RUN: %clang_hwasan %s -DTEST_NO=3 -mllvm -hwasan-instrument-mem-intrinsics -o %t && not %run %t 2>&1 | FileCheck %s --check-prefix=WRITE
 // RUN: %clang_hwasan %s -DTEST_NO=2 -mllvm -hwasan-instrument-mem-intrinsics -o %t && not %env_hwasan_opts=halt_on_error=0 %run %t 2>&1 | FileCheck %s --check-prefix=RECOVER
 
 // REQUIRES: stable-runtime
 
 // Stack aliasing is not implemented on x86.
 // XFAIL: x86_64
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
 
-#include "utils.h"
-
 int main() {
   char Q[16] __attribute__((aligned(256)));
   char P[16] __attribute__((aligned(256)));
 #if TEST_NO == 1
   memset(Q, 0, 32);
 #elif TEST_NO == 2
   memmove(Q, Q + 16, 16);
 #elif TEST_NO == 3
   memcpy(Q, P, 32);
 #endif
-  write(STDOUT_FILENO, UNTAG("recovered\n"), 10);
+  write(STDOUT_FILENO, "recovered\n", 10);
   // WRITE: ERROR: HWAddressSanitizer: tag-mismatch on address
   // WRITE: WRITE of size 32 at {{.*}} tags: [[PTR_TAG:..]]/[[MEM_TAG:..]] (ptr/mem)
   // WRITE: Invalid access starting at offset [16, 32)
   // WRITE: Memory tags around the buggy address (one tag corresponds to 16 bytes):
   // WRITE: =>{{.*}}[[PTR_TAG]]{{[[:space:]]\[}}[[MEM_TAG]]
   // WRITE-NOT: recovered
 
   // READ: ERROR: HWAddressSanitizer: tag-mismatch on address

compiler-rt/test/hwasan/TestCases/set-error-report-callback.cpp

 // RUN: %clangxx_hwasan %s -o %t && not %run %t 2>&1 | FileCheck %s
 
 #include <stdio.h>
 #include <stdlib.h>
 
 #include <sanitizer/hwasan_interface.h>
 
-#include "utils.h"
-
 __attribute__((no_sanitize("hwaddress"))) extern "C" void callback(const char *msg) {
-  untag_fprintf(stderr, "== error start\n%s\n== error end\n", msg);
+  fprintf(stderr, "== error start\n%s\n== error end\n", msg);
 }
 
 int main() {
   __hwasan_enable_allocator_tagging();
   __hwasan_set_error_report_callback(&callback);
   char *volatile p = (char *)malloc(16);
   p[16] = 1;
   free(p);

compiler-rt/test/hwasan/TestCases/sizes.cpp

 #include <string.h>
 
 #include <limits>
 #include <new>
 
 #include <sanitizer/allocator_interface.h>
 #include <sanitizer/hwasan_interface.h>
 
-#include "utils.h"
-
 int main(int argc, char **argv) {
   assert(argc <= 3);
-  bool test_size_max = argc == 3 && !untag_strcmp(argv[2], "max");
+  bool test_size_max = argc == 3 && !strcmp(argv[2], "max");
 
   static const size_t kMaxAllowedMallocSize = 1ULL << 40;
   static const size_t kChunkHeaderSize = 16;
 
   size_t MallocSize = test_size_max ? std::numeric_limits<size_t>::max()
                                     : (kMaxAllowedMallocSize + 1);
 
-  if (!untag_strcmp(argv[1], "malloc")) {
+  if (!strcmp(argv[1], "malloc")) {
     void *p = malloc(MallocSize);
     assert(!p);
-  } else if (!untag_strcmp(argv[1], "calloc")) {
+  } else if (!strcmp(argv[1], "calloc")) {
     // Trigger an overflow in calloc.
     size_t size = std::numeric_limits<size_t>::max();
     void *p = calloc((size / 0x1000) + 1, 0x1000);
     assert(!p);
-  } else if (!untag_strcmp(argv[1], "reallocarray")) {
+  } else if (!strcmp(argv[1], "reallocarray")) {
     // Trigger an overflow in reallocarray.
     size_t size = std::numeric_limits<size_t>::max();
     void *p = __sanitizer_reallocarray(nullptr, (size / 0x1000) + 1, 0x1000);
     assert(!p);
-  } else if (!untag_strcmp(argv[1], "new")) {
+  } else if (!strcmp(argv[1], "new")) {
     void *p = operator new(MallocSize);
     assert(!p);
-  } else if (!untag_strcmp(argv[1], "new-nothrow")) {
+  } else if (!strcmp(argv[1], "new-nothrow")) {
     void *p = operator new(MallocSize, std::nothrow);
     assert(!p);
-  } else if (!untag_strcmp(argv[1], "usable")) {
+  } else if (!strcmp(argv[1], "usable")) {
     // Playing with the actual usable size of a chunk.
     void *p = malloc(1007);
     assert(p);
     size_t size = __sanitizer_get_allocated_size(p);
     assert(size >= 1007);
     memset(p, 'A', size);
     p = realloc(p, 2014);
     assert(p);

compiler-rt/test/hwasan/TestCases/tail-magic.c

 // Tests free_checks_tail_magic=1.
 // RUN: %clang_hwasan  %s -o %t
 // RUN: %env_hwasan_opts=free_checks_tail_magic=0     %run %t
 // RUN: %env_hwasan_opts=free_checks_tail_magic=1 not %run %t 2>&1 | FileCheck %s
 // RUN:                                           not %run %t 2>&1 | FileCheck %s
 
 // REQUIRES: stable-runtime
 
 #include <stdlib.h>
 #include <stdio.h>
 #include <sanitizer/hwasan_interface.h>
 
-#include "utils.h"
-
 static volatile char *sink;
 
 // Overwrite the tail in a non-hwasan function so that we don't detect the
 // stores as OOB.
 __attribute__((no_sanitize("hwaddress"))) void overwrite_tail() {
-  (*UNTAG(&sink))[20] = 0x42;
-  (*UNTAG(&sink))[24] = 0x66;
+  sink[20] = 0x42;
+  sink[24] = 0x66;
 }
 
 int main(int argc, char **argv) {
   __hwasan_enable_allocator_tagging();
 
   char *p = (char*)malloc(20);
-  sink = UNTAG(p);
+  sink = p;
   overwrite_tail();
   free(p);
 // CHECK: ERROR: HWAddressSanitizer: allocation-tail-overwritten; heap object [{{.*}}) of size 20
 // CHECK: in main {{.*}}tail-magic.c:[[@LINE-2]]
 // CHECK: allocated here:
 // CHECK: in main {{.*}}tail-magic.c:[[@LINE-7]]
 // CHECK: Tail contains: .. .. .. .. 42 {{.. .. ..}} 66
 }

compiler-rt/test/hwasan/TestCases/use-after-free.c

 // RUN: %clang_hwasan -O0 -DISREAD=1 %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK
 // RUN: %clang_hwasan -O1 -DISREAD=1 %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK
 // RUN: %clang_hwasan -O2 -DISREAD=1 %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK
 // RUN: %clang_hwasan -O3 -DISREAD=1 %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK
 
 // RUN: %clang_hwasan -O0 -DISREAD=0 %s -o %t && not %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK
 
 // REQUIRES: stable-runtime
 
 #include <stdlib.h>
 #include <stdio.h>
 #include <sanitizer/hwasan_interface.h>
 
-#include "utils.h"
-
 int main() {
   __hwasan_enable_allocator_tagging();
   char * volatile x = (char*)malloc(10);
   free(x);
   __hwasan_disable_allocator_tagging();
-  untag_fprintf(stderr, ISREAD ? "Going to do a READ\n" : "Going to do a WRITE\n");
+  fprintf(stderr, ISREAD ? "Going to do a READ\n" : "Going to do a WRITE\n");
   // CHECK: Going to do a [[TYPE:[A-Z]*]]
   int r = 0;
   if (ISREAD) r = x[5]; else x[5] = 42;  // should be on the same line.
   // CHECK: [[TYPE]] of size 1 at {{.*}} tags: [[PTR_TAG:[0-9a-f][0-9a-f]]]/[[MEM_TAG:[0-9a-f][0-9a-f]]] (ptr/mem)
   // CHECK: #{{[0-9]}} {{.*}} in main {{.*}}use-after-free.c:[[@LINE-2]]
   // Offset is 5 or 11 depending on left/right alignment.
   // CHECK: is a small unallocated heap chunk; size: 32 offset: {{5|11}}
   // CHECK: is located 5 bytes inside of 10-byte region

compiler-rt/test/hwasan/TestCases/utils.h

-#pragma once
-
-#include <stdarg.h>
-#include <stdint.h>
-#include <stdio.h>
-#include <string.h>
-
-#define UNTAG(x) (typeof((x) + 0))(((uintptr_t)(x)) & 0xffffffffffffff)
-
-__attribute__((no_sanitize("hwaddress")))
-int untag_printf(const char *fmt, ...) {
-  va_list ap;
-  va_start(ap, fmt);
-  int ret = vprintf(UNTAG(fmt), ap);
-  va_end(ap);
-  return ret;
-}
-
-__attribute__((no_sanitize("hwaddress")))
-int untag_fprintf(FILE *stream, const char *fmt, ...) {
-  va_list ap;
-  va_start(ap, fmt);
-  int ret = vfprintf(stream, UNTAG(fmt), ap);
-  va_end(ap);
-  return ret;
-}
-
-int untag_strcmp(const char *s1, const char *s2) {
-  return strcmp(UNTAG(s1), UNTAG(s2));
-}

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

 
 static size_t TypeSizeToSizeIndex(uint32_t TypeSize) {
   size_t Res = countTrailingZeros(TypeSize / 8);
   assert(Res < kNumberOfAccessSizes);
   return Res;
 }
 
 void HWAddressSanitizer::untagPointerOperand(Instruction *I, Value *Addr) {
-  if (TargetTriple.isAArch64())
+  if (TargetTriple.isAArch64() || TargetTriple.getArch() == Triple::x86_64)
     return;
 
   IRBuilder<> IRB(I);
   Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   Value *UntaggedPtr =
       IRB.CreateIntToPtr(untagPointer(IRB, AddrLong), Addr->getType());
   I->setOperand(getPointerOperandIndex(I), UntaggedPtr);
 }
if (CompileKernel) {
     Value *ShiftedTag = IRB.CreateShl(Tag, kPointerTagShift);
     TaggedPtrLong = IRB.CreateOr(PtrLong, ShiftedTag);
   }
   return IRB.CreateIntToPtr(TaggedPtrLong, Ty);
 }
 
 // Remove tag from an address.
 Value *HWAddressSanitizer::untagPointer(IRBuilder<> &IRB, Value *PtrLong) {
+  assert(!UsePageAliases);
   Value *UntaggedPtrLong;
   if (CompileKernel) {
     // Kernel addresses have 0xFF in the most significant byte.
     UntaggedPtrLong = IRB.CreateOr(PtrLong,
         ConstantInt::get(PtrLong->getType(), 0xFFULL << kPointerTagShift));
   } else {
     // Userspace addresses have 0x00.
     UntaggedPtrLong = IRB.CreateAnd(PtrLong,

llvm/test/Instrumentation/HWAddressSanitizer/X86/atomic.ll

 ; Test basic address sanitizer instrumentation.
 ;
 ; RUN: opt < %s -hwasan -S | FileCheck %s
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @atomicrmw(i64* %ptr) sanitize_hwaddress {
 ; CHECK-LABEL: @atomicrmw(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i64* %ptr to i64
 
 ; CHECK: call void @__hwasan_store8(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i64* %ptr to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i64*
-; CHECK: atomicrmw add i64* %[[UNTAGGED_PTR]], i64 1 seq_cst
+; CHECK: atomicrmw add i64* %ptr, i64 1 seq_cst
 ; CHECK: ret void
 
 entry:
   %0 = atomicrmw add i64* %ptr, i64 1 seq_cst
   ret void
 }
 
 define void @cmpxchg(i64* %ptr, i64 %compare_to, i64 %new_value) sanitize_hwaddress {
 ; CHECK-LABEL: @cmpxchg(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i64* %ptr to i64
 
 ; CHECK: call void @__hwasan_store8(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i64* %ptr to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i64*
-; CHECK: cmpxchg i64* %[[UNTAGGED_PTR]], i64 %compare_to, i64 %new_value seq_cst seq_cst
+; CHECK: cmpxchg i64* %ptr, i64 %compare_to, i64 %new_value seq_cst seq_cst
 ; CHECK: ret void
 
 entry:
   %0 = cmpxchg i64* %ptr, i64 %compare_to, i64 %new_value seq_cst seq_cst
   ret void
 }

llvm/test/Instrumentation/HWAddressSanitizer/X86/basic.ll

 
 define i8 @test_load8(i8* %a) sanitize_hwaddress {
 ; CHECK-LABEL: @test_load8(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
 
 ; ABORT: call void @__hwasan_load1(i64 %[[A]])
 ; RECOVER: call void @__hwasan_load1_noabort(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i8*
-; CHECK: %[[G:[^ ]*]] = load i8, i8* %[[UNTAGGED_PTR]], align 4
+; CHECK: %[[G:[^ ]*]] = load i8, i8* %a, align 4
 ; CHECK: ret i8 %[[G]]
 
 entry:
   %b = load i8, i8* %a, align 4
   ret i8 %b
 }
 
 define i40 @test_load40(i40* %a) sanitize_hwaddress {
 ; CHECK-LABEL: @test_load40(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
 
 ; ABORT: call void @__hwasan_loadN(i64 %[[A]], i64 5)
 ; RECOVER: call void @__hwasan_loadN_noabort(i64 %[[A]], i64 5)
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i40*
-; CHECK: %[[B:[^ ]*]] = load i40, i40* %[[UNTAGGED_PTR]]
+; CHECK: %[[B:[^ ]*]] = load i40, i40* %a
 ; CHECK: ret i40 %[[B]]
 
 entry:
   %b = load i40, i40* %a, align 4
   ret i40 %b
 }
 
 define void @test_store8(i8* %a, i8 %b) sanitize_hwaddress {
 ; CHECK-LABEL: @test_store8(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
 
 ; ABORT: call void @__hwasan_store1(i64 %[[A]])
 ; RECOVER: call void @__hwasan_store1_noabort(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i8*
-; CHECK: store i8 %b, i8* %[[UNTAGGED_PTR]], align 4
+; CHECK: store i8 %b, i8* %a, align 4
 ; CHECK: ret void
 
 entry:
   store i8 %b, i8* %a, align 4
   ret void
 }
 
 define void @test_store40(i40* %a, i40 %b) sanitize_hwaddress {
 ; CHECK-LABEL: @test_store40(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
 
 ; ABORT: call void @__hwasan_storeN(i64 %[[A]], i64 5)
 ; RECOVER: call void @__hwasan_storeN_noabort(i64 %[[A]], i64 5)
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i40*
-; CHECK: store i40 %b, i40* %[[UNTAGGED_PTR]]
+; CHECK: store i40 %b, i40* %a
 ; CHECK: ret void
 
 entry:
   store i40 %b, i40* %a, align 4
   ret void
 }
 
 define void @test_store_unaligned(i64* %a, i64 %b) sanitize_hwaddress {
 ; CHECK-LABEL: @test_store_unaligned(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i64* %a to i64
 
 ; ABORT: call void @__hwasan_storeN(i64 %[[A]], i64 8)
 ; RECOVER: call void @__hwasan_storeN_noabort(i64 %[[A]], i64 8)
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i64* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i64*
-; CHECK: store i64 %b, i64* %[[UNTAGGED_PTR]], align 4
+; CHECK: store i64 %b, i64* %a, align 4
 ; CHECK: ret void
 
 entry:
   store i64 %b, i64* %a, align 4
   ret void
 }

llvm/test/Instrumentation/HWAddressSanitizer/X86/kernel.ll

 
 define i8 @test_load(i8* %a) sanitize_hwaddress {
 ; CHECK-LABEL: @test_load(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
 
 ; ABORT: call void @__hwasan_load1(i64 %[[A]])
 ; RECOVER: call void @__hwasan_load1_noabort(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = or i64 %[[A]], -72057594037927936
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i8*
-; CHECK: %[[G:[^ ]*]] = load i8, i8* %[[UNTAGGED_PTR]], align 4
+; CHECK: %[[G:[^ ]*]] = load i8, i8* %a, align 4
 ; CHECK: ret i8 %[[G]]
 
 entry:
   %b = load i8, i8* %a, align 4
   ret i8 %b
 }
 
 ; INIT-NOT: call void @__hwasan_init

llvm/test/Instrumentation/HWAddressSanitizer/X86/with-calls.ll

 ; Test basic address sanitizer instrumentation.
 ;
 ; RUN: opt < %s -hwasan -hwasan-instrument-with-calls -S | FileCheck %s --check-prefixes=CHECK,ABORT
 ; RUN: opt < %s -hwasan -hwasan-instrument-with-calls -hwasan-recover=1 -S | FileCheck %s --check-prefixes=CHECK,RECOVER
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define i8 @test_load8(i8* %a) sanitize_hwaddress {
 ; CHECK-LABEL: @test_load8(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
 
 ; ABORT: call void @__hwasan_load1(i64 %[[A]])
 ; RECOVER: call void @__hwasan_load1_noabort(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i8*
-; CHECK: %[[B:[^ ]*]] = load i8, i8* %[[UNTAGGED_PTR]]
+; CHECK: %[[B:[^ ]*]] = load i8, i8* %a
 ; CHECK: ret i8 %[[B]]
 
 entry:
   %b = load i8, i8* %a, align 4
   ret i8 %b
 }
 
 define i40 @test_load40(i40* %a) sanitize_hwaddress {
 ; CHECK-LABEL: @test_load40(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
 
 ; ABORT: call void @__hwasan_loadN(i64 %[[A]], i64 5)
 ; RECOVER: call void @__hwasan_loadN_noabort(i64 %[[A]], i64 5)
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i40*
-; CHECK: %[[B:[^ ]*]] = load i40, i40* %[[UNTAGGED_PTR]]
+; CHECK: %[[B:[^ ]*]] = load i40, i40* %a
 ; CHECK: ret i40 %[[B]]
 
 entry:
   %b = load i40, i40* %a, align 4
   ret i40 %b
 }
 
 define void @test_store8(i8* %a, i8 %b) sanitize_hwaddress {
 ; CHECK-LABEL: @test_store8(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
 
 ; ABORT: call void @__hwasan_store1(i64 %[[A]])
 ; RECOVER: call void @__hwasan_store1_noabort(i64 %[[A]])
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i8* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i8*
-; CHECK: store i8 %b, i8* %[[UNTAGGED_PTR]]
+; CHECK: store i8 %b, i8* %a
 ; CHECK: ret void
 
 entry:
   store i8 %b, i8* %a, align 4
   ret void
 }
 
 define void @test_store40(i40* %a, i40 %b) sanitize_hwaddress {
 ; CHECK-LABEL: @test_store40(
 ; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
 
 ; ABORT: call void @__hwasan_storeN(i64 %[[A]], i64 5)
 ; RECOVER: call void @__hwasan_storeN_noabort(i64 %[[A]], i64 5)
 
-; CHECK: %[[A:[^ ]*]] = ptrtoint i40* %a to i64
-; CHECK: %[[UNTAGGED:[^ ]*]] = and i64 %[[A]], 72057594037927935
-; CHECK: %[[UNTAGGED_PTR:[^ ]*]] = inttoptr i64 %[[UNTAGGED]] to i40*
-; CHECK: store i40 %b, i40* %[[UNTAGGED_PTR]]
+; CHECK: store i40 %b, i40* %a
 ; CHECK: ret void
 
 entry:
   store i40 %b, i40* %a, align 4
   ret void
 }